[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:11 dist-test-slave-x492 krb5kdc[11308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:11 dist-test-slave-x492 krb5kdc[11308](info): set up 2 sockets
Sep 26 07:39:11 dist-test-slave-x492 krb5kdc[11308](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 07:39:12.665432 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:12 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:12.835292 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36337
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:36337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:13.151247 11321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:13.151523 11321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:13.151677 11321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:13.158840 11321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:13.158939 11321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:13.159008 11321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:13.159108 11321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:13.159170 11321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:13.168781 11321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:36337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:13.170113 11321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:13.171506 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:13.187935 11329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:13.187989 11326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:13.188014 11327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:13.188619 11321 server_base.cc:1034] running on GCE node
I20260926 07:39:13.189533 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:13.190897 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:13.192164 11321 hybrid_clock.cc:648] HybridClock initialized: now 1790408353192107 us; error 61 us; skew 500 ppm
I20260926 07:39:13.192577 11321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:13 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:13.195650 11321 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:13.197101 11321 webserver.cc:460] Webserver started at http://127.11.9.126:38355/ using document root <none> and password file <none>
I20260926 07:39:13.197800 11321 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:13.197943 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:13.198247 11321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:13.201541 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "73bfa0e614d64176a9e4118eb6f2fd68"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "41211371ef3a35327fd73d0a15cd2819"
server_key_iv: "714c4f3c03eae30f13759be30966a166"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.202322 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "73bfa0e614d64176a9e4118eb6f2fd68"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "41211371ef3a35327fd73d0a15cd2819"
server_key_iv: "714c4f3c03eae30f13759be30966a166"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.207321 11321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:13.210618 11336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:13.211805 11321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:13.211993 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "73bfa0e614d64176a9e4118eb6f2fd68"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "41211371ef3a35327fd73d0a15cd2819"
server_key_iv: "714c4f3c03eae30f13759be30966a166"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.212190 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:13.235284 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:13.239380 11321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:13.239660 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:13.257642 11321 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:36337
I20260926 07:39:13.257673 11387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:36337 every 8 connection(s)
I20260926 07:39:13.258960 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:13.260995 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11321
I20260926 07:39:13.261372 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:13.263862 11388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:13.263919 11301 external_mini_cluster.cc:1428] Setting key 6b0b395bc5101f1855fd17203fe70233
I20260926 07:39:13.275127 11388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68: Bootstrap starting.
I20260926 07:39:13.278301 11388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:13.279670 11388 log.cc:824] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:13.282464 11388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68: No bootstrap required, opened a new log
I20260926 07:39:13.291486 11388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } }
I20260926 07:39:13.291836 11388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:13.291944 11388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bfa0e614d64176a9e4118eb6f2fd68, State: Initialized, Role: FOLLOWER
I20260926 07:39:13.292534 11388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } }
I20260926 07:39:13.292760 11388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:13.292908 11388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:13.293079 11388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 07:39:13 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:13.295645 11388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } }
I20260926 07:39:13.296211 11388 leader_election.cc:302] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73bfa0e614d64176a9e4118eb6f2fd68; no voters:
I20260926 07:39:13.296963 11388 leader_election.cc:288] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:13.297089 11393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:13.298563 11393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [term 1 LEADER]: Becoming Leader. State: Replica: 73bfa0e614d64176a9e4118eb6f2fd68, State: Running, Role: LEADER
I20260926 07:39:13.299142 11393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } }
I20260926 07:39:13.299458 11388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:13.302219 11394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } } }
I20260926 07:39:13.302559 11394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:13.304710 11395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73bfa0e614d64176a9e4118eb6f2fd68. Latest consensus state: current_term: 1 leader_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bfa0e614d64176a9e4118eb6f2fd68" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36337 } } }
I20260926 07:39:13.305094 11395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:13.306716 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:13.268543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.11.9.126:36337)
0926 07:39:13.269469 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:13.269483 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:13.270792 (+ 1309us) server_negotiation.cc:407] Connection header received
0926 07:39:13.279951 (+ 9159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:13.279976 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:13.280458 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:13.280797 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:13.283629 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.286339 (+ 2710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.288198 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.289350 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.289530 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:13.295663 (+ 6133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:13.295678 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:13.295699 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:13.295740 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:13.300196 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:13.301209 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:13.301220 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:13.301230 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:13.301458 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:13.302112 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:13.302118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:13.302120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:13.302823 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:13.303178 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:13.305440 (+ 2262us) server_negotiation.cc:299] Negotiation successful
0926 07:39:13.305711 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":419,"threads_started":1}
I20260926 07:39:13.309067 11398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:13.316644 11398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:13.325634 11398 catalog_manager.cc:1347] Generated new cluster ID: 3bdbef269ac34ef78668afa148e47253
I20260926 07:39:13.325740 11398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:13.354902 11398 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:13.356204 11398 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:13.371161 11398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73bfa0e614d64176a9e4118eb6f2fd68: Generated new TSK 0
I20260926 07:39:13.372243 11398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:13.425297 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:13.738129 11414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:13.738391 11414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:13.738494 11414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:13.745607 11414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:13.745711 11414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:13.745822 11414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:13.755149 11414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:13.756305 11414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:13.757597 11414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:13.771157 11419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:13.771270 11420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:13.771843 11422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:13.772420 11414 server_base.cc:1034] running on GCE node
I20260926 07:39:13.773072 11414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:13.774066 11414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:13.775316 11414 hybrid_clock.cc:648] HybridClock initialized: now 1790408353775262 us; error 59 us; skew 500 ppm
I20260926 07:39:13.775698 11414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:13 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:13.779042 11414 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:13.780503 11414 webserver.cc:460] Webserver started at http://127.11.9.65:41197/ using document root <none> and password file <none>
I20260926 07:39:13.781220 11414 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:13.781368 11414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:13.781706 11414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:13.784528 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "54483ab090444aa59e87a40de81458e9"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "7d07006983cc856599b5c5e6a312cd6d"
server_key_iv: "88bd543c9c3a98ccdc42ceba0dd4b968"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.785363 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "54483ab090444aa59e87a40de81458e9"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "7d07006983cc856599b5c5e6a312cd6d"
server_key_iv: "88bd543c9c3a98ccdc42ceba0dd4b968"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.790331 11414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:13.793704 11429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:13.795055 11414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:39:13.795261 11414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "54483ab090444aa59e87a40de81458e9"
format_stamp: "Formatted at 2026-09-26 07:39:13 on dist-test-slave-x492"
server_key: "7d07006983cc856599b5c5e6a312cd6d"
server_key_iv: "88bd543c9c3a98ccdc42ceba0dd4b968"
server_key_version: "encryptionkey@0"
I20260926 07:39:13.795433 11414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:13.811630 11414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:13.815958 11414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:13.816247 11414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:13.817260 11414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:13.818480 11414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:13.818553 11414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:13.818696 11414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:13.818773 11414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:13.852803 11414 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:36553
I20260926 07:39:13.852850 11541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:36553 every 8 connection(s)
I20260926 07:39:13.854104 11414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:13.861639 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11414
I20260926 07:39:13.861872 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:13.862509 11301 external_mini_cluster.cc:1428] Setting key 572d2a43a9e6af4fb39fefcc8938e747
Sep 26 07:39:13 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:13.875490 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:13.856708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:51283 (local address 127.11.9.126:36337)
0926 07:39:13.857123 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:13.857134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:13.858114 (+ 980us) server_negotiation.cc:407] Connection header received
0926 07:39:13.859815 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:13.859820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:13.859889 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:13.859986 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:13.861577 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.862401 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.864404 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.864970 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.865103 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:13.869054 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:13.869060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:13.869063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:13.869093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:13.872096 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:13.872876 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:13.872884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:13.872897 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:13.873018 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:13.873524 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:13.873534 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:13.873536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:13.873970 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:13.874136 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:13.874857 (+ 721us) server_negotiation.cc:299] Negotiation successful
0926 07:39:13.875083 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260926 07:39:13.876549 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:13.857283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36337 (local address 127.11.9.65:51283)
0926 07:39:13.857965 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:13.858001 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:13.859563 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:13.860198 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:13.860206 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:13.860702 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:13.861365 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:13.861393 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.862572 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.862580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:13.864265 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:13.864276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:13.865251 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:13.865259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:13.865637 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:13.866446 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:13.866468 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:13.868820 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:13.872262 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:13.872271 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:13.872289 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:13.872736 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:13.873149 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:13.873154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:13.873158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:13.873391 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:13.874107 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:13.874115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:13.874481 (+ 366us) client_negotiation.cc:769] Sending connection context
0926 07:39:13.874759 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 07:39:13.875333 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":261,"threads_started":1}
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:39:13.878304 11542 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36337
I20260926 07:39:13.878623 11542 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:13.879462 11542 heartbeater.cc:511] Master 127.11.9.126:36337 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:39:13.881757 11353 ts_manager.cc:194] Registered new tserver with Master: 54483ab090444aa59e87a40de81458e9 (127.11.9.65:36553)
I20260926 07:39:13.883252 11353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:51283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:13.901610 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:14.217562 11548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:14.217876 11548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:14.218055 11548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:14.225003 11548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:14.225154 11548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:14.225380 11548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:14.234861 11548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:14.236153 11548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:14.237555 11548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:14.250375 11554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:14.250326 11553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:14.250412 11556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:14.251291 11548 server_base.cc:1034] running on GCE node
I20260926 07:39:14.251890 11548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:14.252797 11548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:14.254026 11548 hybrid_clock.cc:648] HybridClock initialized: now 1790408354254006 us; error 134 us; skew 500 ppm
I20260926 07:39:14.254338 11548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:14.257531 11548 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:14.261449 11548 webserver.cc:460] Webserver started at http://127.11.9.66:46239/ using document root <none> and password file <none>
I20260926 07:39:14.262122 11548 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:14.262249 11548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:14.262564 11548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:14.265399 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "58e64116c10a4329b074c278ddd2df3b"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "4531a7d45f53bde9ca01f7de7c3bfafb"
server_key_iv: "6aaea5246ecb492aaab584de868dbfea"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.266129 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "58e64116c10a4329b074c278ddd2df3b"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "4531a7d45f53bde9ca01f7de7c3bfafb"
server_key_iv: "6aaea5246ecb492aaab584de868dbfea"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.271091 11548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260926 07:39:14.274284 11563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.275586 11548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 07:39:14.275750 11548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "58e64116c10a4329b074c278ddd2df3b"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "4531a7d45f53bde9ca01f7de7c3bfafb"
server_key_iv: "6aaea5246ecb492aaab584de868dbfea"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.275931 11548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:14.301664 11548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:14.305789 11548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:14.306033 11548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:14.306882 11548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:14.308084 11548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:14.308152 11548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.308231 11548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:14.308274 11548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.340804 11548 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45715
I20260926 07:39:14.340860 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45715 every 8 connection(s)
I20260926 07:39:14.342127 11548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:14.346513 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11548
I20260926 07:39:14.346704 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:14.347236 11301 external_mini_cluster.cc:1428] Setting key 6f1b8dfe757997c3e02bddf45611d0d1
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:14.363440 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.344816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40635 (local address 127.11.9.126:36337)
0926 07:39:14.344999 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:14.345004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:14.346098 (+ 1094us) server_negotiation.cc:407] Connection header received
0926 07:39:14.348562 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:14.348567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:14.348669 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:14.348784 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:14.350772 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.351474 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.353850 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.354264 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.354398 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.357851 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:14.357864 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:14.357867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:14.357898 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:14.360470 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.361160 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.361204 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.361208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.361334 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.361760 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.361765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.361768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.362093 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:14.362246 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:14.362891 (+ 645us) server_negotiation.cc:299] Negotiation successful
0926 07:39:14.363116 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:39:14.364377 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.345279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36337 (local address 127.11.9.66:40635)
0926 07:39:14.345939 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:14.345979 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:14.348302 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:14.349012 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:14.349020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:14.349762 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:14.350570 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:14.350607 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.351626 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.351634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:14.353699 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:14.353714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.354493 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.354500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:14.354729 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.355401 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:14.355422 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:14.357634 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:14.360612 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:14.360620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:14.360636 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:14.361038 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:14.361449 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:14.361454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:14.361458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:14.361643 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:14.362210 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:14.362219 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:14.362544 (+ 325us) client_negotiation.cc:769] Sending connection context
0926 07:39:14.362794 (+ 250us) client_negotiation.cc:240] Negotiation successful
0926 07:39:14.363296 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":265,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
I20260926 07:39:14.365963 11676 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36337
I20260926 07:39:14.366288 11676 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:14.366983 11676 heartbeater.cc:511] Master 127.11.9.126:36337 requested a full tablet report, sending...
I20260926 07:39:14.368491 11353 ts_manager.cc:194] Registered new tserver with Master: 58e64116c10a4329b074c278ddd2df3b (127.11.9.66:45715)
I20260926 07:39:14.369361 11353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:40635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:14.386489 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:14.699343 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:14.699606 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:14.699766 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:14.706948 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:14.707064 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:14.707288 11681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:14.716733 11681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:14.717988 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:14.719249 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:14.731623 11686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:14.731643 11687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:14.731833 11689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:14.732590 11681 server_base.cc:1034] running on GCE node
I20260926 07:39:14.733137 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:14.734136 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:14.735370 11681 hybrid_clock.cc:648] HybridClock initialized: now 1790408354735314 us; error 49 us; skew 500 ppm
I20260926 07:39:14.735728 11681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:14.738806 11681 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:14.740108 11681 webserver.cc:460] Webserver started at http://127.11.9.67:35287/ using document root <none> and password file <none>
I20260926 07:39:14.740749 11681 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:14.740870 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:14.741251 11681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:14.744038 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "49117d77d3354deda0009a7731ffef32"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "7879a8d3796c1f27d0d69ab2aebd5028"
server_key_iv: "b2898aa1e76174d3529cedbdd10fc4da"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.744789 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "49117d77d3354deda0009a7731ffef32"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "7879a8d3796c1f27d0d69ab2aebd5028"
server_key_iv: "b2898aa1e76174d3529cedbdd10fc4da"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.749804 11681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 07:39:14.753031 11696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.754343 11681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:14.754540 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "49117d77d3354deda0009a7731ffef32"
format_stamp: "Formatted at 2026-09-26 07:39:14 on dist-test-slave-x492"
server_key: "7879a8d3796c1f27d0d69ab2aebd5028"
server_key_iv: "b2898aa1e76174d3529cedbdd10fc4da"
server_key_version: "encryptionkey@0"
I20260926 07:39:14.754743 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:14.782884 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:14.786706 11681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:14.786978 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:14.787878 11681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:14.789117 11681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:14.789240 11681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.789384 11681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:14.789476 11681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:14.821866 11681 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:41573
I20260926 07:39:14.821914 11808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:41573 every 8 connection(s)
I20260926 07:39:14.823194 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:14.831430 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11681
I20260926 07:39:14.831632 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:14.832150 11301 external_mini_cluster.cc:1428] Setting key 525382f95346350dfafcb09884977a02
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:14.843207 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.825763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:37993 (local address 127.11.9.126:36337)
0926 07:39:14.825982 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:14.825988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:14.827109 (+ 1121us) server_negotiation.cc:407] Connection header received
0926 07:39:14.828862 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:14.828866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:14.828953 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:14.829048 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:14.830520 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.831265 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.833552 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.834009 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.834124 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.837948 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:14.837953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:14.837956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:14.837988 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:14.840299 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.840991 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.841001 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.841003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.841118 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.841568 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.841573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.841576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.841836 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:14.841988 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:14.842635 (+ 647us) server_negotiation.cc:299] Negotiation successful
0926 07:39:14.842862 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 07:39:14.844142 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.826280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36337 (local address 127.11.9.67:37993)
0926 07:39:14.826946 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:14.826987 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:14.828585 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:14.829308 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:14.829316 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:14.829793 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:14.830347 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:14.830363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.831429 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.831435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:14.833397 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:14.833411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.834261 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.834269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:14.834587 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.835312 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:14.835333 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:14.837738 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:14.840436 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:14.840444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:14.840460 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:14.840867 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:14.841266 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:14.841271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:14.841275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:14.841450 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:14.841969 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:14.841977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:14.842289 (+ 312us) client_negotiation.cc:769] Sending connection context
0926 07:39:14.842537 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 07:39:14.843057 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":264,"threads_started":1}
I20260926 07:39:14.845570 11809 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36337
I20260926 07:39:14.845847 11809 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:14.846793 11809 heartbeater.cc:511] Master 127.11.9.126:36337 requested a full tablet report, sending...
I20260926 07:39:14.848438 11353 ts_manager.cc:194] Registered new tserver with Master: 49117d77d3354deda0009a7731ffef32 (127.11.9.67:41573)
I20260926 07:39:14.849278 11353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:37993
I20260926 07:39:14.860473 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260926 07:39:14.886236 11542 heartbeater.cc:503] Master 127.11.9.126:36337 was elected leader, sending a full tablet report...
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:14.904446 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.888690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.11.9.65:36553)
0926 07:39:14.889359 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:14.889368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:14.889396 (+ 28us) server_negotiation.cc:407] Connection header received
0926 07:39:14.889640 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:14.889647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:14.889856 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:14.890020 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:14.891425 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.892436 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.893824 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.894377 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.894518 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.898472 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:14.898483 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:14.898497 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:14.898529 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:14.901654 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.902310 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.902317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.902319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.902435 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.902812 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.902821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.902823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.903241 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:14.903540 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:14.903836 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 07:39:14.904098 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":336,"threads_started":1}
W20260926 07:39:14.905709 11457 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47468
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 07:39:14 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:14.947887 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.934390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47480 (local address 127.11.9.65:36553)
0926 07:39:14.934623 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:14.934631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:14.934916 (+ 285us) server_negotiation.cc:407] Connection header received
0926 07:39:14.935127 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:14.935137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:14.935203 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:14.935312 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:14.936690 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.937429 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.938624 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.939049 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.939183 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.942976 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:14.942981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:14.942984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:14.943016 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:14.945449 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.946053 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.946065 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.946068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.946186 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.946550 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.946555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.946559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.946813 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:14.947055 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:14.947357 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 07:39:14.947569 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 07:39:14.949357 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11414
I20260926 07:39:14.961970 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:14.950332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.11.9.65:36553)
0926 07:39:14.950526 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:14.950534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:14.950836 (+ 302us) server_negotiation.cc:407] Connection header received
0926 07:39:14.951001 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:14.951013 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:14.951064 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:14.951162 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:14.952260 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.952948 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.954250 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:14.954675 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:14.954809 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:14.956910 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:14.956915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:14.956918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:14.956946 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:14.959459 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.960058 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.960065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.960068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.960202 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:14.960549 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:14.960555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:14.960558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:14.960805 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:14.961044 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:14.961411 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 07:39:14.961604 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 07:39:14.962798 11537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:15.045924 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11414
I20260926 07:39:15.062816 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11548
Sep 26 07:39:15 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:15.078471 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:15.063689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.11.9.66:45715)
0926 07:39:15.064174 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:15.064181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:15.064204 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:39:15.064261 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:15.064267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:15.064468 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:15.064652 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:15.065896 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:15.066868 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:15.068105 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:15.068656 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:15.068802 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:15.073016 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:15.073027 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:15.073041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:15.073084 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:15.075973 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:15.076633 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:15.076639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:15.076642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:15.076759 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:15.077102 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:15.077105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:15.077107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:15.077513 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:15.077738 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:15.077989 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 07:39:15.078193 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260926 07:39:15.079720 11671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:15.162015 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11548
I20260926 07:39:15.177390 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11681
Sep 26 07:39:15 dist-test-slave-x492 krb5kdc[11308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:15.193327 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:15.178355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.11.9.67:41573)
0926 07:39:15.178987 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:15.178994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:15.179020 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:39:15.179076 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:15.179082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:15.179297 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:15.179467 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:15.180706 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:15.181869 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:15.183137 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:15.183683 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:15.183822 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:15.188051 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:15.188060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:15.188075 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:15.188107 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:15.191043 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:15.191662 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:15.191669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:15.191671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:15.191785 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:15.192083 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:15.192086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:15.192088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:15.192385 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:15.192612 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:15.192799 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 07:39:15.192977 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":344,"threads_started":1}
I20260926 07:39:15.194566 11804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:15.281996 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11681
I20260926 07:39:15.299248 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11321
I20260926 07:39:15.300529 11383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:15.400861 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11321
2026-09-26T07:39:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3827 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:15 dist-test-slave-x492 krb5kdc[11830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:15 dist-test-slave-x492 krb5kdc[11830](info): set up 2 sockets
Sep 26 07:39:15 dist-test-slave-x492 krb5kdc[11830](info): commencing operation
krb5kdc: starting...
W20260926 07:39:16.493314 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:16 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:16.664469 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:43837
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:43837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:16.977419 11843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:16.977697 11843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:16.977867 11843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:16.984858 11843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:16.984959 11843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:16.985064 11843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:16.985162 11843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:16.985275 11843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:16.994800 11843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:43837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:16.996098 11843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:16.997540 11843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:17.009300 11851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:17.009583 11849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:17.009302 11848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:17.010192 11843 server_base.cc:1034] running on GCE node
I20260926 07:39:17.011199 11843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:17.012729 11843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:17.014006 11843 hybrid_clock.cc:648] HybridClock initialized: now 1790408357013951 us; error 61 us; skew 500 ppm
I20260926 07:39:17.014446 11843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:17 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:17.021262 11843 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:17.022864 11843 webserver.cc:460] Webserver started at http://127.11.9.126:33235/ using document root <none> and password file <none>
I20260926 07:39:17.023609 11843 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:17.023761 11843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:17.024096 11843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:17.027285 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "5972ef1735b042efbe5f2d618cf2eca0"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "806c9b6162f526391bb7d53d92fc5827"
server_key_iv: "14146f243f60ca8a058a6d2fb5f06c67"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.028111 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "5972ef1735b042efbe5f2d618cf2eca0"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "806c9b6162f526391bb7d53d92fc5827"
server_key_iv: "14146f243f60ca8a058a6d2fb5f06c67"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.033648 11843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 07:39:17.037490 11858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:17.038841 11843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:17.039083 11843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "5972ef1735b042efbe5f2d618cf2eca0"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "806c9b6162f526391bb7d53d92fc5827"
server_key_iv: "14146f243f60ca8a058a6d2fb5f06c67"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.039341 11843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:17.056916 11843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:17.060993 11843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:17.061324 11843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:17.079499 11843 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:43837
I20260926 07:39:17.079528 11909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:43837 every 8 connection(s)
I20260926 07:39:17.080876 11843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:17.084594 11910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:17.090373 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11843
I20260926 07:39:17.090615 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:17.091437 11301 external_mini_cluster.cc:1428] Setting key aa46b14b48df0c13319dff17b8d6720d
I20260926 07:39:17.096148 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0: Bootstrap starting.
I20260926 07:39:17.099237 11910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:17.100322 11910 log.cc:824] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:17.103574 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0: No bootstrap required, opened a new log
Sep 26 07:39:17 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:17.114632 11910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } }
I20260926 07:39:17.115023 11910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:17.115152 11910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5972ef1735b042efbe5f2d618cf2eca0, State: Initialized, Role: FOLLOWER
I20260926 07:39:17.115831 11910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } }
I20260926 07:39:17.116114 11910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:17.116276 11910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:17.116467 11910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:17.119675 11910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } }
I20260926 07:39:17.120050 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:17.093735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.11.9.126:43837)
0926 07:39:17.094525 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:17.094537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:17.094578 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:39:17.096938 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:17.096963 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:17.097440 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:17.097886 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:17.099390 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.101010 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.102808 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.104161 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.104937 (+ 776us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:17.110733 (+ 5796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:17.110749 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:17.110772 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:17.110838 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:17.115447 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:17.116269 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:17.116277 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:17.116284 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:17.116515 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:17.116954 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:17.116958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:17.116960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:17.117838 (+ 878us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:17.118332 (+ 494us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:17.118649 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 07:39:17.119044 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":363,"threads_started":1}
I20260926 07:39:17.120196 11910 leader_election.cc:302] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5972ef1735b042efbe5f2d618cf2eca0; no voters:
I20260926 07:39:17.120837 11910 leader_election.cc:288] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:17.120968 11915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:17.122797 11910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:17.123610 11915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [term 1 LEADER]: Becoming Leader. State: Replica: 5972ef1735b042efbe5f2d618cf2eca0, State: Running, Role: LEADER
I20260926 07:39:17.124253 11915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } }
I20260926 07:39:17.127293 11917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5972ef1735b042efbe5f2d618cf2eca0. Latest consensus state: current_term: 1 leader_uuid: "5972ef1735b042efbe5f2d618cf2eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } } }
I20260926 07:39:17.127625 11917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:17.128052 11916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5972ef1735b042efbe5f2d618cf2eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ef1735b042efbe5f2d618cf2eca0" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43837 } } }
I20260926 07:39:17.128394 11916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:17.130172 11925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:17.134523 11925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:17.143440 11925 catalog_manager.cc:1347] Generated new cluster ID: 92b1831a7c60424d8fcfc67e15635d24
I20260926 07:39:17.143566 11925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:17.153684 11925 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:17.155427 11925 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:17.161816 11925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5972ef1735b042efbe5f2d618cf2eca0: Generated new TSK 0
I20260926 07:39:17.162744 11925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:17.210947 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:17.523309 11936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:17.523626 11936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:17.523797 11936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:17.530977 11936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:17.531117 11936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:17.531299 11936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:17.540722 11936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:17.541996 11936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:17.543354 11936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:17.555948 11941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:17.555979 11944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:17.555918 11942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:17.556732 11936 server_base.cc:1034] running on GCE node
I20260926 07:39:17.557318 11936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:17.558138 11936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:17.559325 11936 hybrid_clock.cc:648] HybridClock initialized: now 1790408357559272 us; error 49 us; skew 500 ppm
I20260926 07:39:17.559571 11936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:17 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:17.562646 11936 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:17.563997 11936 webserver.cc:460] Webserver started at http://127.11.9.65:33657/ using document root <none> and password file <none>
I20260926 07:39:17.564728 11936 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:17.564872 11936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:17.565217 11936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:17.567963 11936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "8c1803a568e64bc9a008917c213a40cd"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "1ac61e07681578e3d8bd41c97e206f05"
server_key_iv: "9dc648f60d8da6f4ffa04acacd4255f7"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.568727 11936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "8c1803a568e64bc9a008917c213a40cd"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "1ac61e07681578e3d8bd41c97e206f05"
server_key_iv: "9dc648f60d8da6f4ffa04acacd4255f7"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.573468 11936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260926 07:39:17.576838 11951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:17.578132 11936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:17.578315 11936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "8c1803a568e64bc9a008917c213a40cd"
format_stamp: "Formatted at 2026-09-26 07:39:17 on dist-test-slave-x492"
server_key: "1ac61e07681578e3d8bd41c97e206f05"
server_key_iv: "9dc648f60d8da6f4ffa04acacd4255f7"
server_key_version: "encryptionkey@0"
I20260926 07:39:17.578503 11936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:17.595530 11936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:17.599663 11936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:17.599949 11936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:17.600821 11936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:17.602171 11936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:17.602286 11936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:17.602429 11936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:17.602501 11936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:17.635166 11936 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:42459
I20260926 07:39:17.635255 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:42459 every 8 connection(s)
I20260926 07:39:17.636370 11936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:17.636940 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 11936
I20260926 07:39:17.637230 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:17.638078 11301 external_mini_cluster.cc:1428] Setting key 30ec342d423f52c9f2976be3540a452f
Sep 26 07:39:17 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:39:17.660274 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:17.639096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:54141 (local address 127.11.9.126:43837)
0926 07:39:17.639605 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:17.639612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:17.640787 (+ 1175us) server_negotiation.cc:407] Connection header received
0926 07:39:17.643610 (+ 2823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:17.643617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:17.643722 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:17.643865 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:17.645753 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.646439 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.649244 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.649673 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.649767 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:17.654133 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:17.654138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:17.654142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:17.654171 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:17.657287 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:17.657981 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:17.657988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:17.657993 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:17.658134 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:17.658586 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:17.658591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:17.658594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:17.658914 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:17.659057 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:17.659706 (+ 649us) server_negotiation.cc:299] Negotiation successful
0926 07:39:17.659911 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":280,"threads_started":1}
I20260926 07:39:17.661437 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:17.639748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43837 (local address 127.11.9.65:54141)
0926 07:39:17.640628 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:17.640679 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:17.643328 (+ 2649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:17.644109 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:17.644122 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:17.644851 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:17.645595 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:17.645622 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.646590 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.646600 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:17.649044 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:17.649062 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:17.649917 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:17.649927 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:17.650297 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:17.651035 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:17.651084 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:17.653906 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:17.657417 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:17.657426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:17.657444 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:17.657851 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:17.658252 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:17.658258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:17.658262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:17.658437 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:17.659042 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:17.659049 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:17.659373 (+ 324us) client_negotiation.cc:769] Sending connection context
0926 07:39:17.659626 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 07:39:17.660145 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":322,"threads_started":1}
I20260926 07:39:17.663098 12064 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43837
I20260926 07:39:17.663458 12064 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:17.664414 12064 heartbeater.cc:511] Master 127.11.9.126:43837 requested a full tablet report, sending...
I20260926 07:39:17.666673 11875 ts_manager.cc:194] Registered new tserver with Master: 8c1803a568e64bc9a008917c213a40cd (127.11.9.65:42459)
I20260926 07:39:17.668057 11875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:54141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:17.682512 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:17.994910 12070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:17.995206 12070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:17.995383 12070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:18.002550 12070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:18.002660 12070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:18.002842 12070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:18.012578 12070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:18.014024 12070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:18.015311 12070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:18.028694 12075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:18.028666 12076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:18.028615 12078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:18.029464 12070 server_base.cc:1034] running on GCE node
I20260926 07:39:18.030053 12070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:18.030931 12070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:18.032163 12070 hybrid_clock.cc:648] HybridClock initialized: now 1790408358032121 us; error 56 us; skew 500 ppm
I20260926 07:39:18.032475 12070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:18.035929 12070 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:18.037325 12070 webserver.cc:460] Webserver started at http://127.11.9.66:35649/ using document root <none> and password file <none>
I20260926 07:39:18.037999 12070 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:18.038148 12070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:18.038460 12070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:18.041354 12070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "222e084c1f824fd0a4a3b15f9105d01d"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "e477ac4887d73af9859efb1ae5c7185e"
server_key_iv: "940388f4403013dbce438628fc2e8295"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.042135 12070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "222e084c1f824fd0a4a3b15f9105d01d"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "e477ac4887d73af9859efb1ae5c7185e"
server_key_iv: "940388f4403013dbce438628fc2e8295"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.047233 12070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:18.050925 12085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.052193 12070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:18.052385 12070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "222e084c1f824fd0a4a3b15f9105d01d"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "e477ac4887d73af9859efb1ae5c7185e"
server_key_iv: "940388f4403013dbce438628fc2e8295"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.052588 12070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:18.068100 12070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:18.072443 12070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:18.072738 12070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:18.073770 12070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:18.075044 12070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:18.075147 12070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.075309 12070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:18.075381 12070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.117851 12070 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:33937
I20260926 07:39:18.117895 12197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:33937 every 8 connection(s)
I20260926 07:39:18.119042 12070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:18.127656 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12070
I20260926 07:39:18.127849 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:18.128305 11301 external_mini_cluster.cc:1428] Setting key ce5d8662adfd10d3afb4d130cfed3274
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:18.139717 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.121554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:53003 (local address 127.11.9.126:43837)
0926 07:39:18.121724 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.121728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.122932 (+ 1204us) server_negotiation.cc:407] Connection header received
0926 07:39:18.124784 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.124790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.124879 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.124986 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.126575 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.127284 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.128927 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.129455 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.129576 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.133576 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.133581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.133584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.133614 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:18.136523 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.137469 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.137473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.137475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.137610 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.138082 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.138085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.138086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.138345 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:18.138448 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.139297 (+ 849us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.139457 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260926 07:39:18.141225 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.122056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43837 (local address 127.11.9.66:53003)
0926 07:39:18.122784 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.122822 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.124529 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.125282 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.125290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.125775 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:18.126422 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.126440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.127432 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.127437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.128792 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.128803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.129705 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.129710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.129912 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.130698 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:18.130728 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:18.133374 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:18.136664 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:18.136676 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:18.136696 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:18.137338 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:18.137720 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:18.137728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:18.137735 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:18.137968 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:18.138464 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:18.138475 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:18.138914 (+ 439us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.139207 (+ 293us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.139757 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":262,"threads_started":1}
I20260926 07:39:18.143038 12198 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43837
I20260926 07:39:18.143481 12198 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:18.144259 12198 heartbeater.cc:511] Master 127.11.9.126:43837 requested a full tablet report, sending...
I20260926 07:39:18.145906 11875 ts_manager.cc:194] Registered new tserver with Master: 222e084c1f824fd0a4a3b15f9105d01d (127.11.9.66:33937)
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:18.146868 11875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:53003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:18.168829 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:18.477406 12203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:18.477715 12203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:18.477885 12203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:18.484818 12203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:18.484965 12203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:18.485136 12203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:18.494555 12203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:18.495781 12203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:18.497094 12203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:18.509534 12208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:18.509608 12209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:18.509648 12211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:18.510368 12203 server_base.cc:1034] running on GCE node
I20260926 07:39:18.510932 12203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:18.511940 12203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:18.513150 12203 hybrid_clock.cc:648] HybridClock initialized: now 1790408358513099 us; error 47 us; skew 500 ppm
I20260926 07:39:18.513525 12203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:18.516535 12203 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:18.517871 12203 webserver.cc:460] Webserver started at http://127.11.9.67:42363/ using document root <none> and password file <none>
I20260926 07:39:18.518518 12203 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:18.518657 12203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:18.518949 12203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:18.521726 12203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "31552ba6c58440629df5aa39c0e434f9"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "b6d52ec939df8b2ef93860282f702a80"
server_key_iv: "0980a8cb23312dea0b9306915155e0d6"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.522535 12203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "31552ba6c58440629df5aa39c0e434f9"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "b6d52ec939df8b2ef93860282f702a80"
server_key_iv: "0980a8cb23312dea0b9306915155e0d6"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.527590 12203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:18.530786 12218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.532039 12203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 07:39:18.532219 12203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "31552ba6c58440629df5aa39c0e434f9"
format_stamp: "Formatted at 2026-09-26 07:39:18 on dist-test-slave-x492"
server_key: "b6d52ec939df8b2ef93860282f702a80"
server_key_iv: "0980a8cb23312dea0b9306915155e0d6"
server_key_version: "encryptionkey@0"
I20260926 07:39:18.532447 12203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:18.550897 12203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:18.555230 12203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:18.555509 12203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:18.556551 12203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:18.557950 12203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:18.558051 12203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.558198 12203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:18.558269 12203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:18.590554 12203 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:36159
I20260926 07:39:18.590597 12330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:36159 every 8 connection(s)
I20260926 07:39:18.591784 12203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:18.593601 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12203
I20260926 07:39:18.593798 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:18.594264 11301 external_mini_cluster.cc:1428] Setting key 9cff04e313f5a104d3124a02055a00aa
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:18.610991 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.594494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:48787 (local address 127.11.9.126:43837)
0926 07:39:18.594710 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.594716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.596031 (+ 1315us) server_negotiation.cc:407] Connection header received
0926 07:39:18.598204 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.598208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.598271 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.598350 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.599708 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.600400 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.601925 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.602320 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.602411 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.605566 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.605571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.605575 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.605604 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:18.608117 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.608784 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.608791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.608795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.608918 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.609398 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.609403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.609407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.609668 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:18.609821 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.610436 (+ 615us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.610653 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 07:39:18.612025 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.595062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43837 (local address 127.11.9.67:48787)
0926 07:39:18.595883 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.595919 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.597973 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.598547 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.598555 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.599039 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:18.599562 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.599577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.600517 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.600520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.601791 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.601802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.602492 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.602497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.602692 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.603280 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:18.603305 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:18.605378 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:18.608253 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:18.608261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:18.608275 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:18.608656 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:18.609037 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:18.609042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:18.609045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:18.609283 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:18.609798 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:18.609805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:18.610104 (+ 299us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.610355 (+ 251us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.610835 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":357,"threads_started":1}
I20260926 07:39:18.613257 12331 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43837
I20260926 07:39:18.613543 12331 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:18.614158 12331 heartbeater.cc:511] Master 127.11.9.126:43837 requested a full tablet report, sending...
I20260926 07:39:18.615509 11875 ts_manager.cc:194] Registered new tserver with Master: 31552ba6c58440629df5aa39c0e434f9 (127.11.9.67:36159)
I20260926 07:39:18.616233 11875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:48787
I20260926 07:39:18.621363 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:18.639561 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.627345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.11.9.126:43837)
0926 07:39:18.627501 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.627505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.628001 (+ 496us) server_negotiation.cc:407] Connection header received
0926 07:39:18.628115 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.628119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.628173 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.628262 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.629691 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.630402 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.631625 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.631990 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.632079 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.634306 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.634310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.634314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.634340 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:18.636969 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.637663 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.637670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.637673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.637803 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.638196 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.638201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.638204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.638486 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:18.638723 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.639064 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.639276 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 07:39:18.645200 11875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:18.652033 11875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:18.671726 12064 heartbeater.cc:503] Master 127.11.9.126:43837 was elected leader, sending a full tablet report...
I20260926 07:39:18.674343 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.663890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36159 (local address 127.0.0.1:44354)
0926 07:39:18.664933 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.664965 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.665129 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.666483 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.666493 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.666526 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:18.666998 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.667014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.669288 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.669295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.671584 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.671597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.673421 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.673427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.673659 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.673713 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.673815 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.673988 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":650,"threads_started":1}
I20260926 07:39:18.675257 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.665792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:42459 (local address 127.0.0.1:38150)
0926 07:39:18.666828 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.666849 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.666987 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.668499 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.668502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.668520 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:18.668803 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.668810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.670453 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.670457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.672759 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.672773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.674491 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.674497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.674648 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.674682 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.674772 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.674885 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"thread_start_us":678,"threads_started":1}
I20260926 07:39:18.675626 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.664117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.11.9.67:36159)
0926 07:39:18.665799 (+ 1682us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.665808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.665842 (+ 34us) server_negotiation.cc:407] Connection header received
0926 07:39:18.665945 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.665956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.666232 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.667246 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:18.667506 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.669112 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.671783 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.673180 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.674813 (+ 1633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.675040 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.675113 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.675279 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1480,"thread_start_us":1320,"threads_started":1}
I20260926 07:39:18.675946 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.665544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.11.9.65:42459)
0926 07:39:18.667656 (+ 2112us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.667668 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.667712 (+ 44us) server_negotiation.cc:407] Connection header received
0926 07:39:18.667825 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.667835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.668100 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.668290 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:18.669002 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.670312 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.672941 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.674289 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.674432 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.674660 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.675317 (+ 657us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.675484 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1904,"thread_start_us":1730,"threads_started":1}
I20260926 07:39:18.676268 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.664709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:33937 (local address 127.0.0.1:53762)
0926 07:39:18.666029 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.666052 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.666187 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.668906 (+ 2719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.668913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.668939 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:18.669325 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.669341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.671257 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.671273 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.673418 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.673434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.675577 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.675585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.675768 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.675804 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.675873 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.675994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":1034,"threads_started":1}
I20260926 07:39:18.677248 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.665369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.11.9.66:33937)
0926 07:39:18.668410 (+ 3041us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.668418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.668441 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:39:18.668528 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.668536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.668723 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.668899 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:18.669536 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.671099 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.673574 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.675330 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.676477 (+ 1147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.676677 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.676776 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.676927 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2900,"thread_start_us":2796,"threads_started":1}
I20260926 07:39:18.679541 12266 tablet_service.cc:1467] Processing CreateTablet for tablet dca85edf76964eca8554b9fbee10d58a (DEFAULT_TABLE table=test-table [id=e9da2e9ad8f1466784ba3bf494981ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:18.679526 11999 tablet_service.cc:1467] Processing CreateTablet for tablet dca85edf76964eca8554b9fbee10d58a (DEFAULT_TABLE table=test-table [id=e9da2e9ad8f1466784ba3bf494981ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:18.680739 12266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca85edf76964eca8554b9fbee10d58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:18.680831 11999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca85edf76964eca8554b9fbee10d58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:18.680492 12133 tablet_service.cc:1467] Processing CreateTablet for tablet dca85edf76964eca8554b9fbee10d58a (DEFAULT_TABLE table=test-table [id=e9da2e9ad8f1466784ba3bf494981ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:18.681787 12133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca85edf76964eca8554b9fbee10d58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:18.692042 12349 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Bootstrap starting.
I20260926 07:39:18.695272 12350 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Bootstrap starting.
I20260926 07:39:18.695529 12349 tablet_bootstrap.cc:654] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:18.696801 12351 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Bootstrap starting.
I20260926 07:39:18.697192 12349 log.cc:824] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:18.699470 12350 tablet_bootstrap.cc:654] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:18.699940 12351 tablet_bootstrap.cc:654] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:18.700420 12349 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: No bootstrap required, opened a new log
I20260926 07:39:18.700655 12350 log.cc:824] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:18.700695 12349 ts_tablet_manager.cc:1397] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:39:18.700947 12351 log.cc:824] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:18.703562 12350 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: No bootstrap required, opened a new log
I20260926 07:39:18.703727 12351 tablet_bootstrap.cc:492] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: No bootstrap required, opened a new log
I20260926 07:39:18.703821 12350 ts_tablet_manager.cc:1397] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260926 07:39:18.703934 12351 ts_tablet_manager.cc:1397] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260926 07:39:18.708807 12349 raft_consensus.cc:348] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.709206 12349 raft_consensus.cc:374] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:18.709383 12349 raft_consensus.cc:729] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31552ba6c58440629df5aa39c0e434f9, State: Initialized, Role: FOLLOWER
I20260926 07:39:18.709981 12349 consensus_queue.cc:260] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.711030 12331 heartbeater.cc:503] Master 127.11.9.126:43837 was elected leader, sending a full tablet report...
I20260926 07:39:18.711654 12349 ts_tablet_manager.cc:1428] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260926 07:39:18.714419 12351 raft_consensus.cc:348] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.714418 12350 raft_consensus.cc:348] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.714820 12350 raft_consensus.cc:374] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:18.714851 12351 raft_consensus.cc:374] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:18.714938 12351 raft_consensus.cc:729] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1803a568e64bc9a008917c213a40cd, State: Initialized, Role: FOLLOWER
I20260926 07:39:18.714938 12350 raft_consensus.cc:729] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222e084c1f824fd0a4a3b15f9105d01d, State: Initialized, Role: FOLLOWER
I20260926 07:39:18.715529 12351 consensus_queue.cc:260] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.715566 12350 consensus_queue.cc:260] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.717696 12198 heartbeater.cc:503] Master 127.11.9.126:43837 was elected leader, sending a full tablet report...
I20260926 07:39:18.718278 12350 ts_tablet_manager.cc:1428] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260926 07:39:18.718688 12351 ts_tablet_manager.cc:1428] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
W20260926 07:39:18.843387 12332 tablet.cc:2369] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:18.844605 12355 raft_consensus.cc:482] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:18.844874 12355 raft_consensus.cc:504] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.846542 12355 leader_election.cc:288] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1803a568e64bc9a008917c213a40cd (127.11.9.65:42459), 222e084c1f824fd0a4a3b15f9105d01d (127.11.9.66:33937)
I20260926 07:39:18.852093 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.847006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:35263 (local address 127.11.9.66:33937)
0926 07:39:18.847191 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.847197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.847528 (+ 331us) server_negotiation.cc:407] Connection header received
0926 07:39:18.847616 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.847622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.847695 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.847802 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:18.848180 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.848879 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.850499 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.851246 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.851388 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.851476 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.851736 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.851853 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:39:18.852123 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.846866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:33937 (local address 127.11.9.67:35263)
0926 07:39:18.847178 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.847200 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.847369 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.847825 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.847829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.847855 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:18.848045 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.848053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.849021 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.849026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.850343 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.850354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.851412 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.851418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.851584 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.851606 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.851713 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.851836 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":2}
I20260926 07:39:18.852754 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.847000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:37425 (local address 127.11.9.65:42459)
0926 07:39:18.847188 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.847194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.847859 (+ 665us) server_negotiation.cc:407] Connection header received
0926 07:39:18.848087 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.848091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.848148 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.848258 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:18.848782 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.849532 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.851150 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.851904 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.852066 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.852172 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.852458 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.852591 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:39:18.852878 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.846931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:42459 (local address 127.11.9.67:37425)
0926 07:39:18.847743 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:18.847757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:18.847930 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:18.848272 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:18.848276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:18.848297 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:18.848623 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.848631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.849696 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.849701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.851004 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:18.851014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.852107 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.852117 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:18.852285 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.852317 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:39:18.852412 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 07:39:18.852532 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":485,"threads_started":1}
I20260926 07:39:18.853410 12153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca85edf76964eca8554b9fbee10d58a" candidate_uuid: "31552ba6c58440629df5aa39c0e434f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222e084c1f824fd0a4a3b15f9105d01d" is_pre_election: true
I20260926 07:39:18.853924 12153 raft_consensus.cc:2454] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31552ba6c58440629df5aa39c0e434f9 in term 0.
I20260926 07:39:18.853847 12019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca85edf76964eca8554b9fbee10d58a" candidate_uuid: "31552ba6c58440629df5aa39c0e434f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1803a568e64bc9a008917c213a40cd" is_pre_election: true
I20260926 07:39:18.854439 12019 raft_consensus.cc:2454] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31552ba6c58440629df5aa39c0e434f9 in term 0.
I20260926 07:39:18.854604 12219 leader_election.cc:302] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222e084c1f824fd0a4a3b15f9105d01d, 31552ba6c58440629df5aa39c0e434f9; no voters:
I20260926 07:39:18.855007 12355 raft_consensus.cc:2781] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:18.855165 12355 raft_consensus.cc:482] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:18.855270 12355 raft_consensus.cc:3037] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:18.857702 12355 raft_consensus.cc:504] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.858242 12355 leader_election.cc:288] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [CANDIDATE]: Term 1 election: Requested vote from peers 8c1803a568e64bc9a008917c213a40cd (127.11.9.65:42459), 222e084c1f824fd0a4a3b15f9105d01d (127.11.9.66:33937)
I20260926 07:39:18.858675 12019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca85edf76964eca8554b9fbee10d58a" candidate_uuid: "31552ba6c58440629df5aa39c0e434f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1803a568e64bc9a008917c213a40cd"
I20260926 07:39:18.858732 12153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca85edf76964eca8554b9fbee10d58a" candidate_uuid: "31552ba6c58440629df5aa39c0e434f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222e084c1f824fd0a4a3b15f9105d01d"
I20260926 07:39:18.858875 12019 raft_consensus.cc:3037] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:18.858906 12153 raft_consensus.cc:3037] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:18.862021 12019 raft_consensus.cc:2454] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31552ba6c58440629df5aa39c0e434f9 in term 1.
I20260926 07:39:18.862021 12153 raft_consensus.cc:2454] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31552ba6c58440629df5aa39c0e434f9 in term 1.
I20260926 07:39:18.862409 12222 leader_election.cc:302] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31552ba6c58440629df5aa39c0e434f9, 8c1803a568e64bc9a008917c213a40cd; no voters:
I20260926 07:39:18.862697 12355 raft_consensus.cc:2781] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:18.863096 12355 raft_consensus.cc:686] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [term 1 LEADER]: Becoming Leader. State: Replica: 31552ba6c58440629df5aa39c0e434f9, State: Running, Role: LEADER
I20260926 07:39:18.863528 12355 consensus_queue.cc:237] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } }
I20260926 07:39:18.867831 11874 catalog_manager.cc:5495] T dca85edf76964eca8554b9fbee10d58a P 31552ba6c58440629df5aa39c0e434f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31552ba6c58440629df5aa39c0e434f9 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "31552ba6c58440629df5aa39c0e434f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31552ba6c58440629df5aa39c0e434f9" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c1803a568e64bc9a008917c213a40cd" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222e084c1f824fd0a4a3b15f9105d01d" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33937 } health_report { overall_health: UNKNOWN } } }
W20260926 07:39:18.871011 12199 tablet.cc:2369] T dca85edf76964eca8554b9fbee10d58a P 222e084c1f824fd0a4a3b15f9105d01d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:39:18.888896 12065 tablet.cc:2369] T dca85edf76964eca8554b9fbee10d58a P 8c1803a568e64bc9a008917c213a40cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:18.918330 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.903997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.11.9.65:42459)
0926 07:39:18.904221 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.904231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.904694 (+ 463us) server_negotiation.cc:407] Connection header received
0926 07:39:18.904869 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.904873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.904935 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.905035 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.906610 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.907325 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.908610 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.909036 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.909131 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.913153 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.913208 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.913237 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.913271 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:18.915944 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.916539 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.916545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.916547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.916655 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.916971 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.916974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.916976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.917390 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:18.917596 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.917830 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.917978 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:18.961745 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.948470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.11.9.65:42459)
0926 07:39:18.948680 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.948689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.949062 (+ 373us) server_negotiation.cc:407] Connection header received
0926 07:39:18.949286 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.949290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.949354 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.949460 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.950772 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.951513 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.952771 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.953247 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.953375 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.957235 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.957241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.957245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.957287 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:18.959624 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.960157 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.960161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.960163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.960298 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:18.960604 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:18.960607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:18.960609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:18.960856 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:18.961055 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:18.961356 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 07:39:18.961514 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 07:39:18 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:19.003127 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:18.989878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.11.9.65:42459)
0926 07:39:18.990109 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:18.990117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:18.990455 (+ 338us) server_negotiation.cc:407] Connection header received
0926 07:39:18.990663 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:18.990668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:18.990728 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:18.990841 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:18.992112 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.992791 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.994049 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:18.994466 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:18.994596 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:18.998260 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:18.998265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:18.998269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:18.998308 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:19.000580 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.001225 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.001238 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.001241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.001354 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.001737 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.001742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.001746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.002024 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:19.002284 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:19.002555 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 07:39:19.002789 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 07:39:19.004920 11979 tablet_service.cc:2908] Rejecting scan request for tablet dca85edf76964eca8554b9fbee10d58a: Uninitialized: clean time has not yet been initialized
I20260926 07:39:19.007699 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11936
I20260926 07:39:19.019706 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:19.008758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.11.9.65:42459)
0926 07:39:19.008981 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:19.008986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:19.009135 (+ 149us) server_negotiation.cc:407] Connection header received
0926 07:39:19.009332 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:19.009336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:19.009389 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:19.009466 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:19.010540 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.011202 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.012310 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.012831 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.012928 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:19.015020 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:19.015025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:19.015028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:19.015074 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:19.017561 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.018168 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.018173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.018175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.018287 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.018576 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.018580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.018583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.018834 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:19.019053 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:19.019209 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 07:39:19.019367 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:39:19.020366 12059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:19.114565 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11936
I20260926 07:39:19.131695 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12070
Sep 26 07:39:19 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:19.146807 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:19.132577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.11.9.66:33937)
0926 07:39:19.132765 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:19.132770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:19.132789 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:39:19.132950 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:19.132958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:19.133020 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:19.133124 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:19.134290 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.135107 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.136495 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.136891 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.137015 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:19.140946 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:19.140961 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:19.140979 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:19.141020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:19.144280 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.144909 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.144917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.144919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.145051 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.145475 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.145483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.145488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.145865 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:19.146079 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:19.146292 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 07:39:19.146454 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:39:19.147505 12193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:19.235805 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12070
I20260926 07:39:19.252450 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12203
Sep 26 07:39:19 dist-test-slave-x492 krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:19.266746 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:19.253263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.11.9.67:36159)
0926 07:39:19.253681 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:19.253685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:19.253702 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:39:19.253755 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:19.253759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:19.253825 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:19.253923 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:19.255080 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.255739 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.256935 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:19.257461 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:19.257554 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:19.261404 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:19.261418 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:19.261435 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:19.261465 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:19.264024 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.264688 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.264703 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.264706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.264837 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:19.265260 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:19.265265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:19.265268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:19.265594 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:19.265898 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:19.266162 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 07:39:19.266385 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":218,"threads_started":1}
I20260926 07:39:19.267397 12326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:19.356045 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12203
I20260926 07:39:19.372052 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 11843
I20260926 07:39:19.373103 11905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:19.480113 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 11843
2026-09-26T07:39:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4080 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:19 dist-test-slave-x492 krb5kdc[12379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:19 dist-test-slave-x492 krb5kdc[12379](info): set up 2 sockets
Sep 26 07:39:19 dist-test-slave-x492 krb5kdc[12379](info): commencing operation
krb5kdc: starting...
W20260926 07:39:20.573392 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:20 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:20.740872 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39781
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:21.051441 12392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:21.051792 12392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:21.051954 12392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:21.059345 12392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:21.059474 12392 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 07:39:21.059577 12392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:21.059684 12392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:21.059777 12392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:21.059867 12392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:21.069607 12392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:39781
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:21.070883 12392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:21.072242 12392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:21.083698 12398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:21.083638 12397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:21.083868 12400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:21.084474 12392 server_base.cc:1034] running on GCE node
I20260926 07:39:21.085318 12392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:21.086597 12392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:21.087806 12392 hybrid_clock.cc:648] HybridClock initialized: now 1790408361087746 us; error 33 us; skew 500 ppm
I20260926 07:39:21.088125 12392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:21 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408361, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:21.091161 12392 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:21.092518 12392 webserver.cc:460] Webserver started at http://127.11.9.126:41971/ using document root <none> and password file <none>
I20260926 07:39:21.093266 12392 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:21.093401 12392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:21.093719 12392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:21.096485 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "fbe494af7d274b4fae9c0d8777f0bd2f"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "2f28eb41543b5394666f282324a90fd7"
server_key_iv: "233b97cee5c02ee7c2b2ce7d894ddc02"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.097321 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "fbe494af7d274b4fae9c0d8777f0bd2f"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "2f28eb41543b5394666f282324a90fd7"
server_key_iv: "233b97cee5c02ee7c2b2ce7d894ddc02"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.102149 12392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 07:39:21.105432 12407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:21.106658 12392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:39:21.106842 12392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "fbe494af7d274b4fae9c0d8777f0bd2f"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "2f28eb41543b5394666f282324a90fd7"
server_key_iv: "233b97cee5c02ee7c2b2ce7d894ddc02"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.107046 12392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:21.127326 12392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:21.131553 12392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:21.131811 12392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:21.153677 12392 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:39781
I20260926 07:39:21.153719 12468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:39781 every 8 connection(s)
I20260926 07:39:21.155054 12392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:21.156426 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12392
I20260926 07:39:21.156668 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:21.157509 11301 external_mini_cluster.cc:1428] Setting key 0502c16b7e1179be4c4502090e8325fd
I20260926 07:39:21.159794 12469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 07:39:21 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:21.172748 12469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Bootstrap starting.
I20260926 07:39:21.176913 12469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:21.178141 12469 log.cc:824] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:21.181439 12469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: No bootstrap required, opened a new log
I20260926 07:39:21.181632 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:21.159475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.11.9.126:39781)
0926 07:39:21.160193 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:21.160208 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:21.160257 (+ 49us) server_negotiation.cc:407] Connection header received
0926 07:39:21.162581 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:21.162607 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:21.163120 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:21.163543 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:21.164782 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.166828 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.168018 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.169011 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.169216 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:21.173278 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:21.173287 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:21.173305 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:21.173344 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:21.177280 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.177919 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.177930 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.177943 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.178135 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.178535 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.178541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.178543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.179195 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:21.179405 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:21.179975 (+ 570us) server_negotiation.cc:299] Negotiation successful
0926 07:39:21.180368 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":316,"threads_started":1}
I20260926 07:39:21.189316 12469 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } }
I20260926 07:39:21.189613 12469 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:21.189703 12469 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe494af7d274b4fae9c0d8777f0bd2f, State: Initialized, Role: FOLLOWER
I20260926 07:39:21.190284 12469 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } }
I20260926 07:39:21.190507 12469 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:21.190660 12469 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:21.190824 12469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:21.192795 12469 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } }
I20260926 07:39:21.193361 12469 leader_election.cc:302] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe494af7d274b4fae9c0d8777f0bd2f; no voters:
I20260926 07:39:21.193876 12469 leader_election.cc:288] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:21.194025 12474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:21.194697 12474 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [term 1 LEADER]: Becoming Leader. State: Replica: fbe494af7d274b4fae9c0d8777f0bd2f, State: Running, Role: LEADER
I20260926 07:39:21.195525 12474 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } }
I20260926 07:39:21.195971 12469 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:21.198139 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe494af7d274b4fae9c0d8777f0bd2f. Latest consensus state: current_term: 1 leader_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } } }
I20260926 07:39:21.198514 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:21.199153 12475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe494af7d274b4fae9c0d8777f0bd2f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39781 } } }
I20260926 07:39:21.199637 12475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:21.203250 12483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:21.207276 12483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:21.215711 12483 catalog_manager.cc:1347] Generated new cluster ID: 0a2b6852bfb844b19be35ad238468717
I20260926 07:39:21.215840 12483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:21.225276 12483 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:21.227025 12483 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:21.237905 12483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Generated new TSK 0
I20260926 07:39:21.238848 12483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:21.284538 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39781
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:21.598006 12496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:21.598294 12496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:21.598460 12496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:21.605458 12496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:21.605595 12496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:21.605741 12496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:21.605880 12496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:21.615324 12496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:21.616510 12496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:21.617871 12496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:21.630745 12501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:21.630719 12502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:21.630950 12504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:21.631973 12496 server_base.cc:1034] running on GCE node
I20260926 07:39:21.632524 12496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:21.633394 12496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:21.634600 12496 hybrid_clock.cc:648] HybridClock initialized: now 1790408361634543 us; error 31 us; skew 500 ppm
I20260926 07:39:21.634893 12496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:21 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408361, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:21.637981 12496 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:21.639415 12496 webserver.cc:460] Webserver started at http://127.11.9.65:37981/ using document root <none> and password file <none>
I20260926 07:39:21.640112 12496 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:21.640218 12496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:21.640527 12496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:21.643306 12496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "6f065c310322416baad4109f5830ba3e"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "7a3cf1e51528e7f2a6fc21e5b63efa18"
server_key_iv: "6c01a018779eee71131b49ede7d74dae"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.644074 12496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "6f065c310322416baad4109f5830ba3e"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "7a3cf1e51528e7f2a6fc21e5b63efa18"
server_key_iv: "6c01a018779eee71131b49ede7d74dae"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.648880 12496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:39:21.652042 12511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:21.653342 12496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:21.653494 12496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "6f065c310322416baad4109f5830ba3e"
format_stamp: "Formatted at 2026-09-26 07:39:21 on dist-test-slave-x492"
server_key: "7a3cf1e51528e7f2a6fc21e5b63efa18"
server_key_iv: "6c01a018779eee71131b49ede7d74dae"
server_key_version: "encryptionkey@0"
I20260926 07:39:21.653693 12496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:21.666327 12496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:21.670418 12496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:21.670691 12496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:21.673121 12496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:21.673255 12496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 07:39:21.673458 12496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:21.673560 12496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:39:21 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408361, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:21.695433 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:21.673522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:60765 (local address 127.11.9.126:39781)
0926 07:39:21.673709 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:21.673715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:21.675036 (+ 1321us) server_negotiation.cc:407] Connection header received
0926 07:39:21.677196 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:21.677209 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:21.677300 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:21.677407 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:21.679542 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.680545 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.682989 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.683502 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.683649 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:21.688694 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:21.688702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:21.688707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:21.688748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:21.692110 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.693099 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.693108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.693114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.693316 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.693811 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.693819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.693823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.694102 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:21.694246 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:21.694883 (+ 637us) server_negotiation.cc:299] Negotiation successful
0926 07:39:21.695103 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:39:21.696568 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:21.674019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.65:60765)
0926 07:39:21.674849 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:21.674901 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:21.676915 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:21.677653 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:21.677668 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:21.678455 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:21.679317 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:21.679344 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.680685 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.680691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:21.682771 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:21.682782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.683757 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.683765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:21.684065 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:21.684875 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:21.684927 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:21.688435 (+ 3508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:21.692283 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:21.692294 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:21.692315 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:21.692953 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:21.693457 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:21.693462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:21.693468 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:21.693675 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:21.694244 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:21.694252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:21.694554 (+ 302us) client_negotiation.cc:769] Sending connection context
0926 07:39:21.694803 (+ 249us) client_negotiation.cc:240] Negotiation successful
0926 07:39:21.695288 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":366,"threads_started":1}
I20260926 07:39:21.717530 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:21.702430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:42227 (local address 127.11.9.126:39781)
0926 07:39:21.702633 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:21.702642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:21.703213 (+ 571us) server_negotiation.cc:407] Connection header received
0926 07:39:21.703344 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:21.703352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:21.703445 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:21.703549 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:21.704365 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.705721 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.706997 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.707979 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.708136 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:21.709933 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:21.709943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:21.709950 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:21.709997 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:21.714057 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.715258 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.715264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.715268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.715404 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:21.715898 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:21.715905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:21.715908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:21.716217 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:21.716386 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:21.716848 (+ 462us) server_negotiation.cc:299] Negotiation successful
0926 07:39:21.717087 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:39:21.717669 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:21.702478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.65:42227)
0926 07:39:21.703102 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:21.703121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:21.703232 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:21.703582 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:21.703585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:21.703987 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:21.704238 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:21.704246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.705895 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.705899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:21.706865 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:21.706876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:21.708155 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:21.708159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:21.708283 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:21.708891 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:21.708906 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:21.709724 (+ 818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:21.714258 (+ 4534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:21.714270 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:21.714277 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:21.715034 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:21.715587 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:21.715594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:21.715598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:21.715784 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:21.716354 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:21.716360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:21.716569 (+ 209us) client_negotiation.cc:769] Sending connection context
0926 07:39:21.716752 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 07:39:21.717087 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":348,"threads_started":1}
I20260926 07:39:21.732517 12496 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:43493
I20260926 07:39:21.732524 12630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:43493 every 8 connection(s)
I20260926 07:39:21.733745 12496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:21.735790 12631 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39781
I20260926 07:39:21.736004 12631 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:21.736629 12631 heartbeater.cc:511] Master 127.11.9.126:39781 requested a full tablet report, sending...
I20260926 07:39:21.738536 12424 ts_manager.cc:194] Registered new tserver with Master: 6f065c310322416baad4109f5830ba3e (127.11.9.65:43493)
I20260926 07:39:21.740044 12424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:60765
I20260926 07:39:21.741603 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12496
I20260926 07:39:21.741811 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:21.742587 11301 external_mini_cluster.cc:1428] Setting key 5016dbcf3f02cdd88cd60bcf9c14d032
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:21.779822 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39781
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:22.097378 12635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:22.097648 12635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:22.097841 12635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:22.105137 12635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:22.105257 12635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:22.105396 12635 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:22.105554 12635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:22.116058 12635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:22.117822 12635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:22.119663 12635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:22.133685 12640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:22.133639 12643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:22.133653 12641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:22.134361 12635 server_base.cc:1034] running on GCE node
I20260926 07:39:22.134981 12635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:22.135936 12635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:22.137205 12635 hybrid_clock.cc:648] HybridClock initialized: now 1790408362137099 us; error 45 us; skew 500 ppm
I20260926 07:39:22.137467 12635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:22.148602 12635 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:22.150081 12635 webserver.cc:460] Webserver started at http://127.11.9.66:33287/ using document root <none> and password file <none>
I20260926 07:39:22.150784 12635 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:22.150898 12635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:22.151227 12635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:22.154186 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "1c7a88fe8e4448a583a1101399064228"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "ab97f7d573b4694923dbae865a8f1a17"
server_key_iv: "5ea6889cfcdc32dd217c960b88ff4547"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.155046 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "1c7a88fe8e4448a583a1101399064228"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "ab97f7d573b4694923dbae865a8f1a17"
server_key_iv: "5ea6889cfcdc32dd217c960b88ff4547"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.160285 12635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:39:22.163751 12650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:22.165107 12635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:22.165313 12635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "1c7a88fe8e4448a583a1101399064228"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "ab97f7d573b4694923dbae865a8f1a17"
server_key_iv: "5ea6889cfcdc32dd217c960b88ff4547"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.165581 12635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:22.179531 12635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:22.183235 12635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:22.183517 12635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:22.186230 12635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:22.186339 12635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:22.186522 12635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:22.186596 12635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:22.210093 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.186465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:44963 (local address 127.11.9.126:39781)
0926 07:39:22.186661 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.186669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.187977 (+ 1308us) server_negotiation.cc:407] Connection header received
0926 07:39:22.190026 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.190031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.190135 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.190241 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:22.192274 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.193629 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.196249 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.197006 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.197152 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.203040 (+ 5888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:22.203049 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:22.203057 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:22.203109 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:22.206864 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.207797 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.207802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.207808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.207937 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.208351 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.208357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.208361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.208640 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:22.208790 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.209518 (+ 728us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.209736 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 07:39:22.211198 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.186935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.66:44963)
0926 07:39:22.187797 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.187849 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.189711 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.190459 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.190466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.191212 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:22.191992 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.192010 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.193816 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.193824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.196040 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.196051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.197367 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.197376 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.197712 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.198576 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:22.198614 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:22.202721 (+ 4107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:22.206987 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.206999 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.207015 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.207672 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.208043 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.208049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.208053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.208238 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.208769 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:22.208786 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:22.209100 (+ 314us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.209450 (+ 350us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.209964 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":401,"threads_started":1}
I20260926 07:39:22.231041 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.217850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40381 (local address 127.11.9.126:39781)
0926 07:39:22.218099 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.218107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.218659 (+ 552us) server_negotiation.cc:407] Connection header received
0926 07:39:22.218852 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.218857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.218919 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.219012 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:22.219869 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.220659 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.222398 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.223028 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.223167 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.224843 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:22.224849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:22.224854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:22.224892 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:22.228149 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.229043 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.229049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.229055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.229231 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.229590 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.229598 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.229601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.229904 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:22.230051 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.230474 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.230675 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:39:22.231379 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.217785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.66:40381)
0926 07:39:22.218558 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.218583 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.218718 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.219032 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.219037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.219473 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:22.219737 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.219746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.220806 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.220812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.222122 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.222133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.223225 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.223232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.223408 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.224000 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:22.224015 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:22.224665 (+ 650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:22.228291 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.228297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.228302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.228904 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.229347 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.229356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.229360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.229482 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.230023 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:22.230027 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:22.230209 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.230383 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.230709 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":498,"threads_started":1}
I20260926 07:39:22.240850 12635 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:33387
I20260926 07:39:22.240887 12769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:33387 every 8 connection(s)
I20260926 07:39:22.242081 12635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:22.243922 12770 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39781
I20260926 07:39:22.244141 12770 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:22.244747 12770 heartbeater.cc:511] Master 127.11.9.126:39781 requested a full tablet report, sending...
I20260926 07:39:22.246038 12424 ts_manager.cc:194] Registered new tserver with Master: 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387)
I20260926 07:39:22.246317 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12635
I20260926 07:39:22.246505 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:22.247022 11301 external_mini_cluster.cc:1428] Setting key 81bdddff599e436309f184ac70a5303d
I20260926 07:39:22.247140 12424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:44963
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:22.284456 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39781
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:22.600001 12774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:22.600303 12774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:22.600466 12774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:22.607415 12774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:22.607558 12774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:22.607707 12774 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:22.607839 12774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:22.617733 12774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:39781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:22.618929 12774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:22.620256 12774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:22.633189 12782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:22.633322 12779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:22.633226 12780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:22.633919 12774 server_base.cc:1034] running on GCE node
I20260926 07:39:22.634506 12774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:22.635357 12774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:22.636641 12774 hybrid_clock.cc:648] HybridClock initialized: now 1790408362636550 us; error 39 us; skew 500 ppm
I20260926 07:39:22.636947 12774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:22.640038 12774 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:22.641490 12774 webserver.cc:460] Webserver started at http://127.11.9.67:39875/ using document root <none> and password file <none>
I20260926 07:39:22.642135 12774 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:22.642261 12774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:22.642558 12774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:22.645301 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "a9540a0d200749268be71f23d78bddab"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "de11808e153034c4a65e47d715c9f2ff"
server_key_iv: "2dcc299ab558a5e3ef2700ad8d5b9472"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.646129 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "a9540a0d200749268be71f23d78bddab"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "de11808e153034c4a65e47d715c9f2ff"
server_key_iv: "2dcc299ab558a5e3ef2700ad8d5b9472"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.650911 12774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:39:22.654084 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:22.655295 12774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:39:22.655467 12774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "a9540a0d200749268be71f23d78bddab"
format_stamp: "Formatted at 2026-09-26 07:39:22 on dist-test-slave-x492"
server_key: "de11808e153034c4a65e47d715c9f2ff"
server_key_iv: "2dcc299ab558a5e3ef2700ad8d5b9472"
server_key_version: "encryptionkey@0"
I20260926 07:39:22.655671 12774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:22.669690 12774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:22.673767 12774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:22.674031 12774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:22.676553 12774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:22.676679 12774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:22.676860 12774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:22.676939 12774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:22.699985 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.676963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:60195 (local address 127.11.9.126:39781)
0926 07:39:22.677140 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.677146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.678556 (+ 1410us) server_negotiation.cc:407] Connection header received
0926 07:39:22.680545 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.680551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.680637 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.680746 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:22.682627 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.683696 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.687007 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.687599 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.687733 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.692983 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:22.692987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:22.692990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:22.693021 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:22.696337 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.697116 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.697121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.697125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.697296 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.697909 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.697914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.697918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.698268 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:22.698425 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.699505 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.699689 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 07:39:22.701953 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.677537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.67:60195)
0926 07:39:22.678379 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.678423 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.680284 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.680971 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.680979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.681617 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:22.682434 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.682466 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.683872 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.683882 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.686840 (+ 2958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.686862 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.688030 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.688040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.688340 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.689075 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:22.689099 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:22.692731 (+ 3632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:22.696475 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.696483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.696499 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.696992 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.697415 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.697424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.697427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.697744 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.698410 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:22.698422 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:22.698960 (+ 538us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.699412 (+ 452us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.700073 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":398,"threads_started":1}
I20260926 07:39:22.723117 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.707398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:58757 (local address 127.11.9.126:39781)
0926 07:39:22.707603 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.707610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.708280 (+ 670us) server_negotiation.cc:407] Connection header received
0926 07:39:22.708485 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.708489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.708553 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.708630 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:22.710115 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.710903 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.713395 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.714013 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.714133 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.716636 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:22.716641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:22.716644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:22.716671 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:22.719693 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.720665 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.720671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.720676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.720825 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.721421 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.721428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.721433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.721816 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:22.721991 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.722541 (+ 550us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.722744 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:39:22.723330 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.707383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.11.9.67:58757)
0926 07:39:22.708137 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.708176 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.708369 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.708694 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.708701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.709485 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:22.709968 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.709985 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.711033 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.711041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.713206 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.713228 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.714224 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.714232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.714527 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.715170 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:22.715201 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:22.716437 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:22.719820 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.719829 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.719835 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.720539 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.721004 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:22.721012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:22.721018 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:22.721233 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:22.721965 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:22.721976 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:22.722207 (+ 231us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.722418 (+ 211us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.722840 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":378,"threads_started":1}
I20260926 07:39:22.734855 12774 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:36139
I20260926 07:39:22.734875 12908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:36139 every 8 connection(s)
I20260926 07:39:22.736083 12774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:22.738199 12909 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39781
I20260926 07:39:22.738386 12909 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:22.738952 12909 heartbeater.cc:511] Master 127.11.9.126:39781 requested a full tablet report, sending...
I20260926 07:39:22.739897 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 12774
I20260926 07:39:22.740078 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:22.740200 12424 ts_manager.cc:194] Registered new tserver with Master: a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:22.740535 11301 external_mini_cluster.cc:1428] Setting key f43baaa43f1a1eee8c746dfd3fe3d8d5
I20260926 07:39:22.741055 12424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:60195
I20260926 07:39:22.743237 12631 heartbeater.cc:503] Master 127.11.9.126:39781 was elected leader, sending a full tablet report...
I20260926 07:39:22.743526 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 07:39:22 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:22.786813 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.773971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.11.9.126:39781)
0926 07:39:22.774193 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.774200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.774362 (+ 162us) server_negotiation.cc:407] Connection header received
0926 07:39:22.774517 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.774520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.774590 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.774674 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:22.776042 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.776763 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.778004 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.778403 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.778492 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.782279 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:22.782284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:22.782287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:22.782329 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:22.784559 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.785117 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.785120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.785123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.785352 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:22.785683 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:22.785686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:22.785688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:22.785909 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:22.786079 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.786326 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.786480 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 07:39:22.790902 12423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:22.797770 12423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:22.822548 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.810780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43493 (local address 127.0.0.1:59534)
0926 07:39:22.813701 (+ 2921us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.813742 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.813904 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.814728 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.814736 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.814774 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:22.815084 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.815092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.816955 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.816962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.819494 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.819508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.821400 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.821407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.821675 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.821725 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.821913 (+ 188us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.822100 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2675,"mutex_wait_us":1,"thread_start_us":2561,"threads_started":1}
I20260926 07:39:22.823248 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.811170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.11.9.65:43493)
0926 07:39:22.812761 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.812767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.813833 (+ 1066us) server_negotiation.cc:407] Connection header received
0926 07:39:22.814148 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.814156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.814478 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.814696 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:22.815240 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.816815 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.819688 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.821227 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.821359 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.821626 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.822772 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.822945 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":1225,"threads_started":1}
I20260926 07:39:22.823989 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.815083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.11.9.67:36139)
0926 07:39:22.815792 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.815800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.816459 (+ 659us) server_negotiation.cc:407] Connection header received
0926 07:39:22.816639 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.816644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.816806 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.816991 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:22.817997 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.819001 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.820746 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.822076 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.822191 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.822407 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.823501 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.823682 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":359,"threads_started":1}
I20260926 07:39:22.824170 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.811234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.11.9.66:33387)
0926 07:39:22.812336 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.812349 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.814575 (+ 2226us) server_negotiation.cc:407] Connection header received
0926 07:39:22.814786 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.814795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.815032 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.815194 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:22.816004 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.817350 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.821067 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.822770 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.822865 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.823065 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.823738 (+ 673us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.823879 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":783,"threads_started":1}
I20260926 07:39:22.824393 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.814533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36139 (local address 127.0.0.1:58604)
0926 07:39:22.816355 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.816368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.816494 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.817267 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.817272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.817302 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:22.817792 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.817801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.819162 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.819167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.820606 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.820617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.822558 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.822566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.822755 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.822785 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.822867 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.822977 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710,"mutex_wait_us":20,"thread_start_us":1592,"threads_started":1}
I20260926 07:39:22.824818 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.811173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:33387 (local address 127.0.0.1:55540)
0926 07:39:22.814462 (+ 3289us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.814478 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.814585 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.815470 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.815474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.815498 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:22.815845 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.815858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.817493 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.817499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.820899 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.820913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.823247 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.823252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.823508 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.823542 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.823681 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.823807 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3161,"mutex_wait_us":65,"thread_start_us":3028,"threads_started":1}
I20260926 07:39:22.827289 12561 tablet_service.cc:1467] Processing CreateTablet for tablet cefe315a39244c04a3cab674e294a555 (DEFAULT_TABLE table=test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:22.827505 12839 tablet_service.cc:1467] Processing CreateTablet for tablet cefe315a39244c04a3cab674e294a555 (DEFAULT_TABLE table=test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:22.827924 12700 tablet_service.cc:1467] Processing CreateTablet for tablet cefe315a39244c04a3cab674e294a555 (DEFAULT_TABLE table=test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:22.828650 12839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cefe315a39244c04a3cab674e294a555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:22.828691 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cefe315a39244c04a3cab674e294a555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:22.829052 12700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cefe315a39244c04a3cab674e294a555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:22.840157 12928 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Bootstrap starting.
I20260926 07:39:22.842635 12929 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Bootstrap starting.
I20260926 07:39:22.843926 12928 tablet_bootstrap.cc:654] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:22.845260 12928 log.cc:824] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:22.846407 12929 tablet_bootstrap.cc:654] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:22.846611 12930 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Bootstrap starting.
I20260926 07:39:22.847625 12929 log.cc:824] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:22.848412 12928 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: No bootstrap required, opened a new log
I20260926 07:39:22.848687 12928 ts_tablet_manager.cc:1397] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 07:39:22.849862 12930 tablet_bootstrap.cc:654] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:22.850538 12929 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: No bootstrap required, opened a new log
I20260926 07:39:22.850792 12929 ts_tablet_manager.cc:1397] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 07:39:22.850975 12930 log.cc:824] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:22.854063 12930 tablet_bootstrap.cc:492] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: No bootstrap required, opened a new log
I20260926 07:39:22.854295 12930 ts_tablet_manager.cc:1397] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 07:39:22.856295 12928 raft_consensus.cc:348] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.856698 12928 raft_consensus.cc:374] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:22.856823 12928 raft_consensus.cc:729] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9540a0d200749268be71f23d78bddab, State: Initialized, Role: FOLLOWER
I20260926 07:39:22.857533 12928 consensus_queue.cc:260] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.858606 12909 heartbeater.cc:503] Master 127.11.9.126:39781 was elected leader, sending a full tablet report...
I20260926 07:39:22.859236 12928 ts_tablet_manager.cc:1428] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260926 07:39:22.861109 12929 raft_consensus.cc:348] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.861573 12929 raft_consensus.cc:374] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:22.861660 12929 raft_consensus.cc:729] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Initialized, Role: FOLLOWER
I20260926 07:39:22.862269 12929 consensus_queue.cc:260] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.863937 12929 ts_tablet_manager.cc:1428] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260926 07:39:22.865831 12930 raft_consensus.cc:348] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.866205 12930 raft_consensus.cc:374] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:22.866325 12930 raft_consensus.cc:729] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7a88fe8e4448a583a1101399064228, State: Initialized, Role: FOLLOWER
I20260926 07:39:22.866870 12930 consensus_queue.cc:260] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.867825 12770 heartbeater.cc:503] Master 127.11.9.126:39781 was elected leader, sending a full tablet report...
I20260926 07:39:22.868304 12930 ts_tablet_manager.cc:1428] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260926 07:39:22.970842 12935 raft_consensus.cc:482] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:22.971175 12935 raft_consensus.cc:504] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.972851 12935 leader_election.cc:288] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:22.979497 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.973401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:34539 (local address 127.11.9.66:33387)
0926 07:39:22.973592 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.973598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.974004 (+ 406us) server_negotiation.cc:407] Connection header received
0926 07:39:22.974328 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.974337 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.974439 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.974501 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:22.975285 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.975995 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.977828 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.978561 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.978711 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.978824 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.979107 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.979236 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:39:22.979774 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.974184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:54829 (local address 127.11.9.67:36139)
0926 07:39:22.974483 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:22.974489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:22.974783 (+ 294us) server_negotiation.cc:407] Connection header received
0926 07:39:22.974999 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:22.975004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:22.975070 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:22.975168 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:22.975664 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.976345 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.978148 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.978883 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.979033 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.979143 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:22.979417 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 07:39:22.979531 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:39:22.979877 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.973140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:33387 (local address 127.11.9.65:34539)
0926 07:39:22.973912 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.973931 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.974133 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.974758 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.974764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.974792 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:22.975118 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.975127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.976123 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.976131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.977664 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.977675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.978774 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.978779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.978947 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.978969 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.979088 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.979209 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":25,"thread_start_us":353,"threads_started":1}
I20260926 07:39:22.979899 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:22.973961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36139 (local address 127.11.9.65:54829)
0926 07:39:22.974675 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:22.974692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:22.974848 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:22.975182 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:22.975186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:22.975208 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:22.975525 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.975533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.976508 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.976513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.978003 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:22.978015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:22.979108 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:22.979114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:22.979261 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:22.979283 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:39:22.979395 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 07:39:22.979513 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":24,"thread_start_us":409,"threads_started":1}
I20260926 07:39:22.980928 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cefe315a39244c04a3cab674e294a555" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228" is_pre_election: true
I20260926 07:39:22.980957 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cefe315a39244c04a3cab674e294a555" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab" is_pre_election: true
I20260926 07:39:22.981451 12724 raft_consensus.cc:2454] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:22.981451 12864 raft_consensus.cc:2454] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:22.982091 12514 leader_election.cc:302] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a88fe8e4448a583a1101399064228, 6f065c310322416baad4109f5830ba3e; no voters:
I20260926 07:39:22.982451 12935 raft_consensus.cc:2781] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:22.982611 12935 raft_consensus.cc:482] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:22.982720 12935 raft_consensus.cc:3037] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:22.984812 12935 raft_consensus.cc:504] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:22.985394 12935 leader_election.cc:288] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Requested vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:22.985862 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cefe315a39244c04a3cab674e294a555" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228"
W20260926 07:39:22.986019 12632 tablet.cc:2369] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:22.986027 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cefe315a39244c04a3cab674e294a555" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab"
I20260926 07:39:22.986080 12724 raft_consensus.cc:3037] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:22.986243 12864 raft_consensus.cc:3037] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:39:22.987679 12910 tablet.cc:2369] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:22.988433 12724 raft_consensus.cc:2454] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:22.988816 12514 leader_election.cc:302] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a88fe8e4448a583a1101399064228, 6f065c310322416baad4109f5830ba3e; no voters:
I20260926 07:39:22.989089 12935 raft_consensus.cc:2781] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:22.989228 12864 raft_consensus.cc:2454] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:22.989683 12935 raft_consensus.cc:686] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 1 LEADER]: Becoming Leader. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Running, Role: LEADER
I20260926 07:39:22.990139 12935 consensus_queue.cc:237] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
W20260926 07:39:22.993891 12771 tablet.cc:2369] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:22.994762 12423 catalog_manager.cc:5495] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f065c310322416baad4109f5830ba3e (127.11.9.65). New cstate: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:23.029814 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.024447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.11.9.65:43493)
0926 07:39:23.024787 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.024791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.024813 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:39:23.025024 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.025028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.025111 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.025283 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:23.025601 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.026285 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.027527 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.028136 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.028229 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.028719 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:23.029240 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:23.029359 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.029451 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.029510 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260926 07:39:23.037366 12864 raft_consensus.cc:1261] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.037428 12724 raft_consensus.cc:1261] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.038393 12939 consensus_queue.cc:1035] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.038714 12935 consensus_queue.cc:1035] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.052337 12943 mvcc.cc:205] Tried to move back new op lower bound from 7333512654991319040 to 7333512654810775552. Current Snapshot: MvccSnapshot[applied={T|T < 7333512654991319040}]
I20260926 07:39:23.067443 12424 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:23.067673 12424 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:23.071345 12424 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Sending DeleteTablet for 3 replicas of tablet cefe315a39244c04a3cab674e294a555
I20260926 07:39:23.072445 12561 tablet_service.cc:1514] Processing DeleteTablet for tablet cefe315a39244c04a3cab674e294a555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:59534
I20260926 07:39:23.072732 12839 tablet_service.cc:1514] Processing DeleteTablet for tablet cefe315a39244c04a3cab674e294a555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:58604
I20260926 07:39:23.072916 12700 tablet_service.cc:1514] Processing DeleteTablet for tablet cefe315a39244c04a3cab674e294a555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:55540
I20260926 07:39:23.077708 12424 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:23.078553 12424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:23.091081 12699 tablet_service.cc:1467] Processing CreateTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (DEFAULT_TABLE table=test-table [id=7bec8bcb6f6a4120929938eae511b10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:23.091150 12838 tablet_service.cc:1467] Processing CreateTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (DEFAULT_TABLE table=test-table [id=7bec8bcb6f6a4120929938eae511b10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:23.091816 12699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ee74b09c7f44a1a2a76ab8c37daa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.091918 12838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ee74b09c7f44a1a2a76ab8c37daa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.094295 12560 tablet_service.cc:1467] Processing CreateTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (DEFAULT_TABLE table=test-table [id=7bec8bcb6f6a4120929938eae511b10e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:23.095144 12560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ee74b09c7f44a1a2a76ab8c37daa66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.104347 12928 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Bootstrap starting.
I20260926 07:39:23.106681 12928 tablet_bootstrap.cc:654] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.118357 12929 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Bootstrap starting.
I20260926 07:39:23.118829 12930 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Bootstrap starting.
I20260926 07:39:23.120075 12929 tablet_bootstrap.cc:654] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.120669 12930 tablet_bootstrap.cc:654] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.121776 12928 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: No bootstrap required, opened a new log
I20260926 07:39:23.121965 12928 ts_tablet_manager.cc:1397] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.009s
I20260926 07:39:23.123212 12928 raft_consensus.cc:348] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.123534 12928 raft_consensus.cc:374] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.123659 12928 raft_consensus.cc:729] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9540a0d200749268be71f23d78bddab, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.124061 12928 consensus_queue.cc:260] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.128105 12928 ts_tablet_manager.cc:1428] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260926 07:39:23.132845 12956 tablet_replica.cc:330] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: stopping tablet replica
I20260926 07:39:23.133463 12956 raft_consensus.cc:2229] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:23.134023 12956 raft_consensus.cc:2258] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.134279 12930 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: No bootstrap required, opened a new log
I20260926 07:39:23.134534 12930 ts_tablet_manager.cc:1397] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.008s
I20260926 07:39:23.135464 12930 raft_consensus.cc:348] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.135859 12930 raft_consensus.cc:374] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.136044 12930 raft_consensus.cc:729] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7a88fe8e4448a583a1101399064228, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.136452 12930 consensus_queue.cc:260] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.139233 12956 ts_tablet_manager.cc:1905] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.139559 12930 ts_tablet_manager.cc:1428] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260926 07:39:23.144276 12954 tablet_replica.cc:330] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: stopping tablet replica
I20260926 07:39:23.144785 12954 raft_consensus.cc:2229] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:23.145459 12929 tablet_bootstrap.cc:492] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: No bootstrap required, opened a new log
I20260926 07:39:23.145541 12954 raft_consensus.cc:2258] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.145757 12929 ts_tablet_manager.cc:1397] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.009s
I20260926 07:39:23.146430 12955 tablet_replica.cc:330] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: stopping tablet replica
I20260926 07:39:23.146780 12929 raft_consensus.cc:348] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.147063 12929 raft_consensus.cc:374] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.147208 12929 raft_consensus.cc:729] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.147483 12929 consensus_queue.cc:260] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.148047 12929 ts_tablet_manager.cc:1428] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 07:39:23.152153 12956 ts_tablet_manager.cc:1918] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:23.152315 12956 log.cc:1196] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/cefe315a39244c04a3cab674e294a555
I20260926 07:39:23.152815 12956 ts_tablet_manager.cc:1939] T cefe315a39244c04a3cab674e294a555 P 1c7a88fe8e4448a583a1101399064228: Deleting consensus metadata
I20260926 07:39:23.146909 12955 raft_consensus.cc:2229] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:23.153662 12955 raft_consensus.cc:2258] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.155546 12954 ts_tablet_manager.cc:1905] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.157873 12955 ts_tablet_manager.cc:1905] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.163118 12955 ts_tablet_manager.cc:1918] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:23.163296 12955 log.cc:1196] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/cefe315a39244c04a3cab674e294a555
I20260926 07:39:23.163827 12955 ts_tablet_manager.cc:1939] T cefe315a39244c04a3cab674e294a555 P a9540a0d200749268be71f23d78bddab: Deleting consensus metadata
I20260926 07:39:23.165742 12410 catalog_manager.cc:4841] TS a9540a0d200749268be71f23d78bddab (127.11.9.67:36139): tablet cefe315a39244c04a3cab674e294a555 (table test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]) successfully deleted
I20260926 07:39:23.167307 12410 catalog_manager.cc:4841] TS 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387): tablet cefe315a39244c04a3cab674e294a555 (table test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]) successfully deleted
I20260926 07:39:23.171288 12954 ts_tablet_manager.cc:1918] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:23.171494 12954 log.cc:1196] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/cefe315a39244c04a3cab674e294a555
I20260926 07:39:23.172049 12954 ts_tablet_manager.cc:1939] T cefe315a39244c04a3cab674e294a555 P 6f065c310322416baad4109f5830ba3e: Deleting consensus metadata
I20260926 07:39:23.173576 12409 catalog_manager.cc:4841] TS 6f065c310322416baad4109f5830ba3e (127.11.9.65:43493): tablet cefe315a39244c04a3cab674e294a555 (table test-table [id=1100afa86b1e4f0d9a94160ef05cf8fc]) successfully deleted
I20260926 07:39:23.284265 12949 raft_consensus.cc:482] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:23.284569 12949 raft_consensus.cc:504] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.285560 12949 leader_election.cc:288] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:23.286145 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ee74b09c7f44a1a2a76ab8c37daa66" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228" is_pre_election: true
I20260926 07:39:23.286221 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ee74b09c7f44a1a2a76ab8c37daa66" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab" is_pre_election: true
I20260926 07:39:23.286412 12724 raft_consensus.cc:2454] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:23.286484 12864 raft_consensus.cc:2454] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:23.286887 12514 leader_election.cc:302] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a88fe8e4448a583a1101399064228, 6f065c310322416baad4109f5830ba3e; no voters:
I20260926 07:39:23.287194 12949 raft_consensus.cc:2781] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:23.287336 12949 raft_consensus.cc:482] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:23.287452 12949 raft_consensus.cc:3037] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.290892 12949 raft_consensus.cc:504] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.291558 12949 leader_election.cc:288] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Requested vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:23.292048 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ee74b09c7f44a1a2a76ab8c37daa66" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab"
I20260926 07:39:23.292040 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ee74b09c7f44a1a2a76ab8c37daa66" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228"
I20260926 07:39:23.292241 12864 raft_consensus.cc:3037] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.292255 12724 raft_consensus.cc:3037] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.294173 12864 raft_consensus.cc:2454] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:23.294592 12514 leader_election.cc:302] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f065c310322416baad4109f5830ba3e, a9540a0d200749268be71f23d78bddab; no voters:
I20260926 07:39:23.294863 12949 raft_consensus.cc:2781] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:23.295019 12949 raft_consensus.cc:686] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 1 LEADER]: Becoming Leader. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Running, Role: LEADER
I20260926 07:39:23.295418 12724 raft_consensus.cc:2454] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:23.295310 12949 consensus_queue.cc:237] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.298316 12424 catalog_manager.cc:5495] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f065c310322416baad4109f5830ba3e (127.11.9.65). New cstate: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } health_report { overall_health: UNKNOWN } } }
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408361, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:23.343348 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.327366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39781 (local address 127.0.0.1:50184)
0926 07:39:23.329559 (+ 2193us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:23.329582 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:23.329718 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:23.330436 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:23.330440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:23.331007 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:23.331190 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.331198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.332603 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.332607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.333560 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.333571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.334463 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.334468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.334623 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.335299 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:23.335318 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:23.337663 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:23.341140 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:23.341149 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:23.341154 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:23.341584 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:23.341949 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:23.341952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:23.341954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:23.342095 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:23.342627 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:23.342632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:23.342762 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 07:39:23.342908 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 07:39:23.343082 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2035,"mutex_wait_us":32,"thread_start_us":1944,"threads_started":1}
I20260926 07:39:23.343691 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.327402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.11.9.126:39781)
0926 07:39:23.330131 (+ 2729us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.330137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.330154 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:39:23.330204 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.330209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.330274 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.330388 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.331340 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.332483 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.333689 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.334324 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.334449 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.337853 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.337861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.337867 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.337902 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.341014 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.341704 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.341709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.341713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.341851 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.342198 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.342202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.342206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.342515 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.342649 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.343032 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.343219 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":2617,"thread_start_us":2523,"threads_started":1}
I20260926 07:39:23.348384 12424 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:50184:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 07:39:23.349476 12424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:23.355907 12561 tablet_service.cc:1467] Processing CreateTablet for tablet 2f758ca58109460cab75ae02b687b21e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e9fc4144bf44fe912ec9d013eaf851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:23.355969 12700 tablet_service.cc:1467] Processing CreateTablet for tablet 2f758ca58109460cab75ae02b687b21e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e9fc4144bf44fe912ec9d013eaf851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:23.356552 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f758ca58109460cab75ae02b687b21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.356600 12700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f758ca58109460cab75ae02b687b21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.359692 12839 tablet_service.cc:1467] Processing CreateTablet for tablet 2f758ca58109460cab75ae02b687b21e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3e9fc4144bf44fe912ec9d013eaf851]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:23.360225 12839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f758ca58109460cab75ae02b687b21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:23.363457 12930 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: Bootstrap starting.
I20260926 07:39:23.365077 12930 tablet_bootstrap.cc:654] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.366114 12929 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: Bootstrap starting.
I20260926 07:39:23.367089 12928 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: Bootstrap starting.
I20260926 07:39:23.367831 12929 tablet_bootstrap.cc:654] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.368615 12928 tablet_bootstrap.cc:654] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:23.371171 12929 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: No bootstrap required, opened a new log
I20260926 07:39:23.371356 12929 ts_tablet_manager.cc:1397] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260926 07:39:23.372169 12928 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: No bootstrap required, opened a new log
I20260926 07:39:23.372185 12929 raft_consensus.cc:348] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.372380 12928 ts_tablet_manager.cc:1397] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260926 07:39:23.372490 12929 raft_consensus.cc:374] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.372596 12929 raft_consensus.cc:729] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.372867 12929 consensus_queue.cc:260] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.373334 12949 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.373363 12928 raft_consensus.cc:348] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.373617 12949 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.373624 12930 tablet_bootstrap.cc:492] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: No bootstrap required, opened a new log
I20260926 07:39:23.373761 12928 raft_consensus.cc:374] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.373767 12930 ts_tablet_manager.cc:1397] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260926 07:39:23.373901 12928 raft_consensus.cc:729] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9540a0d200749268be71f23d78bddab, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.374212 12928 consensus_queue.cc:260] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.374502 12930 raft_consensus.cc:348] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.374722 12930 raft_consensus.cc:374] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:23.374789 12930 raft_consensus.cc:729] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7a88fe8e4448a583a1101399064228, State: Initialized, Role: FOLLOWER
I20260926 07:39:23.374687 12934 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.374943 12934 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.375025 12930 consensus_queue.cc:260] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.375895 12936 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.376224 12936 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.376233 12928 ts_tablet_manager.cc:1428] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 07:39:23.376446 12930 ts_tablet_manager.cc:1428] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260926 07:39:23.378496 12929 ts_tablet_manager.cc:1428] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260926 07:39:23.398914 12949 raft_consensus.cc:482] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:23.399135 12949 raft_consensus.cc:504] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.399938 12949 leader_election.cc:288] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:23.400441 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f758ca58109460cab75ae02b687b21e" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab" is_pre_election: true
I20260926 07:39:23.400466 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f758ca58109460cab75ae02b687b21e" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228" is_pre_election: true
I20260926 07:39:23.400691 12864 raft_consensus.cc:2454] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:23.400753 12724 raft_consensus.cc:2454] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 0.
I20260926 07:39:23.401245 12514 leader_election.cc:302] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7a88fe8e4448a583a1101399064228, 6f065c310322416baad4109f5830ba3e; no voters:
I20260926 07:39:23.401561 12949 raft_consensus.cc:2781] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:23.401669 12949 raft_consensus.cc:482] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:23.401798 12949 raft_consensus.cc:3037] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.405031 12949 raft_consensus.cc:504] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.405721 12949 leader_election.cc:288] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Requested vote from peers 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387), a9540a0d200749268be71f23d78bddab (127.11.9.67:36139)
I20260926 07:39:23.406104 12724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f758ca58109460cab75ae02b687b21e" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7a88fe8e4448a583a1101399064228"
I20260926 07:39:23.406213 12864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f758ca58109460cab75ae02b687b21e" candidate_uuid: "6f065c310322416baad4109f5830ba3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9540a0d200749268be71f23d78bddab"
I20260926 07:39:23.406330 12724 raft_consensus.cc:3037] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.406450 12864 raft_consensus.cc:3037] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:23.409224 12724 raft_consensus.cc:2454] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:23.409230 12864 raft_consensus.cc:2454] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f065c310322416baad4109f5830ba3e in term 1.
I20260926 07:39:23.409653 12514 leader_election.cc:302] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f065c310322416baad4109f5830ba3e, a9540a0d200749268be71f23d78bddab; no voters:
I20260926 07:39:23.409996 12949 raft_consensus.cc:2781] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:23.410390 12949 raft_consensus.cc:686] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [term 1 LEADER]: Becoming Leader. State: Replica: 6f065c310322416baad4109f5830ba3e, State: Running, Role: LEADER
I20260926 07:39:23.410701 12949 consensus_queue.cc:237] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } }
I20260926 07:39:23.412679 12952 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: TxnStatusTablet state changed. Reason: New leader 6f065c310322416baad4109f5830ba3e. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.413053 12952 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:23.413838 12972 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 07:39:23.414075 12972 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 07:39:23.414088 12423 catalog_manager.cc:5495] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f065c310322416baad4109f5830ba3e (127.11.9.65). New cstate: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:23.432957 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.427395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43493 (local address 127.0.0.1:59556)
0926 07:39:23.427695 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:23.427710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:23.427819 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:23.428213 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:23.428217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:23.428226 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:23.428368 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.428374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.429909 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.429913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.430864 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.430874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.431861 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.431865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.432044 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.432093 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:23.432551 (+ 458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:23.432563 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 07:39:23.432624 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 07:39:23.432727 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260926 07:39:23.433041 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.427368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.11.9.65:43493)
0926 07:39:23.427532 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.427536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.427790 (+ 254us) server_negotiation.cc:407] Connection header received
0926 07:39:23.427987 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.427993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.428083 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.428192 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:23.428499 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.429780 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.431017 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.431708 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.431837 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.432277 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:23.432446 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:23.432547 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.432672 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.432761 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:39:23.436568 12864 raft_consensus.cc:1261] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.436560 12724 raft_consensus.cc:1261] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.437222 12949 consensus_queue.cc:1035] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.437557 12952 consensus_queue.cc:1035] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.441619 12936 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: TxnStatusTablet state changed. Reason: New leader 6f065c310322416baad4109f5830ba3e. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.441970 12936 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.444339 12949 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.444650 12949 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:23.444825 12944 mvcc.cc:205] Tried to move back new op lower bound from 7333512656630177792 to 7333512656533098496. Current Snapshot: MvccSnapshot[applied={T|T < 7333512656630177792 or (T in {7333512656630177792})}]
I20260926 07:39:23.445338 12936 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.445676 12936 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 1c7a88fe8e4448a583a1101399064228: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.456537 12934 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: TxnStatusTablet state changed. Reason: New leader 6f065c310322416baad4109f5830ba3e. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.456924 12934 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.459699 12952 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.459950 12952 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P 6f065c310322416baad4109f5830ba3e: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:23.460968 12934 tablet_replica.cc:439] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f065c310322416baad4109f5830ba3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f065c310322416baad4109f5830ba3e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43493 } } peers { permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 } } peers { permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 } } }
I20260926 07:39:23.461226 12934 tablet_replica.cc:442] T 2f758ca58109460cab75ae02b687b21e P a9540a0d200749268be71f23d78bddab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:23.473471 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.467227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:48063 (local address 127.11.9.65:43493)
0926 07:39:23.467379 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.467383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.468090 (+ 707us) server_negotiation.cc:407] Connection header received
0926 07:39:23.468332 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.468339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.468440 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.468548 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:23.469095 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.470039 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.471700 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.472139 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.472276 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.472702 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:23.472822 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:23.472925 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.473043 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.473108 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":48}
I20260926 07:39:23.473670 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.467344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43493 (local address 127.11.9.65:48063)
0926 07:39:23.467973 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:23.467992 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:23.468196 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:23.468574 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:23.468578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:23.468589 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:23.468945 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.468953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.470176 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.470181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.471563 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:23.471576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.472325 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.472333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:23.472464 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.472511 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:23.472939 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:23.472951 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 07:39:23.473019 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 07:39:23.473144 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":366,"threads_started":1}
I20260926 07:39:23.482683 12864 raft_consensus.cc:1261] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.482726 12724 raft_consensus.cc:1261] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Refusing update from remote peer 6f065c310322416baad4109f5830ba3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:23.483312 12949 consensus_queue.cc:1035] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9540a0d200749268be71f23d78bddab" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.483644 12935 consensus_queue.cc:1035] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7a88fe8e4448a583a1101399064228" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:23.541558 12423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:23.541810 12423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:23.544551 12423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe494af7d274b4fae9c0d8777f0bd2f: Sending DeleteTablet for 3 replicas of tablet 12ee74b09c7f44a1a2a76ab8c37daa66
I20260926 07:39:23.545454 12700 tablet_service.cc:1514] Processing DeleteTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:55540
I20260926 07:39:23.545483 12839 tablet_service.cc:1514] Processing DeleteTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:58604
I20260926 07:39:23.545480 12560 tablet_service.cc:1514] Processing DeleteTablet for tablet 12ee74b09c7f44a1a2a76ab8c37daa66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:23 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:59534
I20260926 07:39:23.545742 12954 tablet_replica.cc:330] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: stopping tablet replica
I20260926 07:39:23.545743 12955 tablet_replica.cc:330] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: stopping tablet replica
I20260926 07:39:23.545928 12955 raft_consensus.cc:2229] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:23.545928 12954 raft_consensus.cc:2229] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:23.546090 12955 raft_consensus.cc:2258] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.546226 12954 raft_consensus.cc:2258] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.546341 12956 tablet_replica.cc:330] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: stopping tablet replica
I20260926 07:39:23.546552 12956 raft_consensus.cc:2229] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:23.546741 12956 raft_consensus.cc:2258] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:23.547447 12955 ts_tablet_manager.cc:1905] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.547919 12954 ts_tablet_manager.cc:1905] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.555696 12954 ts_tablet_manager.cc:1918] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:23.556573 12956 ts_tablet_manager.cc:1905] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:23.557243 12955 ts_tablet_manager.cc:1918] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:23.557513 12955 log.cc:1196] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/12ee74b09c7f44a1a2a76ab8c37daa66
I20260926 07:39:23.558053 12955 ts_tablet_manager.cc:1939] T 12ee74b09c7f44a1a2a76ab8c37daa66 P a9540a0d200749268be71f23d78bddab: Deleting consensus metadata
I20260926 07:39:23.559412 12410 catalog_manager.cc:4841] TS a9540a0d200749268be71f23d78bddab (127.11.9.67:36139): tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (table test-table [id=7bec8bcb6f6a4120929938eae511b10e]) successfully deleted
I20260926 07:39:23.560523 12954 log.cc:1196] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/12ee74b09c7f44a1a2a76ab8c37daa66
I20260926 07:39:23.561050 12954 ts_tablet_manager.cc:1939] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 6f065c310322416baad4109f5830ba3e: Deleting consensus metadata
I20260926 07:39:23.562438 12409 catalog_manager.cc:4841] TS 6f065c310322416baad4109f5830ba3e (127.11.9.65:43493): tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (table test-table [id=7bec8bcb6f6a4120929938eae511b10e]) successfully deleted
I20260926 07:39:23.563927 12956 ts_tablet_manager.cc:1918] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:23.564081 12956 log.cc:1196] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/12ee74b09c7f44a1a2a76ab8c37daa66
I20260926 07:39:23.564563 12956 ts_tablet_manager.cc:1939] T 12ee74b09c7f44a1a2a76ab8c37daa66 P 1c7a88fe8e4448a583a1101399064228: Deleting consensus metadata
I20260926 07:39:23.565589 12410 catalog_manager.cc:4841] TS 1c7a88fe8e4448a583a1101399064228 (127.11.9.66:33387): tablet 12ee74b09c7f44a1a2a76ab8c37daa66 (table test-table [id=7bec8bcb6f6a4120929938eae511b10e]) successfully deleted
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:23.578521 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.560588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.11.9.65:43493)
0926 07:39:23.560848 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.560856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.563269 (+ 2413us) server_negotiation.cc:407] Connection header received
0926 07:39:23.563471 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.563476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.563556 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.563669 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.565263 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.566210 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.567466 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.567961 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.568113 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.572542 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.572557 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.572572 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.572605 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.575649 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.576310 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.576330 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.576332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.576467 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.576872 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.576877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.576882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.577347 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.577682 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.577977 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.578196 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260926 07:39:23.579182 12626 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59566
I20260926 07:39:23.594841 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.581804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.11.9.126:39781)
0926 07:39:23.581985 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.581990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.582313 (+ 323us) server_negotiation.cc:407] Connection header received
0926 07:39:23.582509 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.582514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.582589 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.582686 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.583812 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.584870 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.586083 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.586767 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.586923 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.589002 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.589008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.589013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.589050 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.592236 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.592856 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.592861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.592865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.593014 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.593436 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.593442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.593447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.593792 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.594035 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.594276 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.594498 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 07:39:23.595515 12423 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50200
I20260926 07:39:23.597533 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12496
I20260926 07:39:23.616251 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.598271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.11.9.65:43493)
0926 07:39:23.599490 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.599499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.603516 (+ 4017us) server_negotiation.cc:407] Connection header received
0926 07:39:23.603614 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.603620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.603697 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.603802 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.605011 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.606163 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.607310 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.607943 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.608070 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.610304 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.610311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.610316 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.610352 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.613924 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.614537 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.614543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.614547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.614675 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.614972 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.614977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.614981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.615304 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.615549 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.615714 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.615908 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045}
W20260926 07:39:23.616854 12626 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59576
W20260926 07:39:23.617406 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12496: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:23.617591 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12496
I20260926 07:39:23.639521 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12635
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:23.653441 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.640322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.11.9.66:33387)
0926 07:39:23.640876 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.640880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.640905 (+ 25us) server_negotiation.cc:407] Connection header received
0926 07:39:23.640956 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.640960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.641023 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.641113 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.642277 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.642982 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.644097 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.644489 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.644579 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.648376 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.648388 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.648406 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.648434 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.651281 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.651825 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.651831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.651834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.651976 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.652275 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.652278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.652280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.652581 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.652776 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.652979 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.653127 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":303,"threads_started":1}
W20260926 07:39:23.654044 12765 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55544
W20260926 07:39:23.654580 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12635: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:23.654744 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12635
I20260926 07:39:23.671177 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12774
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408362, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:23.684839 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:23.671949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.11.9.67:36139)
0926 07:39:23.672433 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:23.672438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:23.672454 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:39:23.672504 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:23.672507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:23.672560 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:23.672652 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:23.673825 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.674519 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.675660 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:23.676062 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:23.676149 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:23.680050 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:23.680062 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:23.680078 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:23.680111 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:23.682676 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.683258 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.683265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.683267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.683388 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:23.683675 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:23.683678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:23.683680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:23.683997 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:23.684194 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:23.684442 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 07:39:23.684587 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":256,"threads_started":1}
W20260926 07:39:23.685554 12904 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58608
W20260926 07:39:23.686188 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:23.686338 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12774
I20260926 07:39:23.702665 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 12392
I20260926 07:39:23.703677 12464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:23.818823 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 12392
2026-09-26T07:39:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4340 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12992](info): set up 2 sockets
Sep 26 07:39:23 dist-test-slave-x492 krb5kdc[12992](info): commencing operation
krb5kdc: starting...
W20260926 07:39:24.905309 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:25 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:25.100006 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42791
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:25.411599 13005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:25.411875 13005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:25.411991 13005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:25.419092 13005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:25.419189 13005 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 07:39:25.419234 13005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:25.419315 13005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:25.419387 13005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:25.419490 13005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:25.429224 13005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42791
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:25.430480 13005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:25.431833 13005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:25.443244 13011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:25.443266 13013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:25.443370 13010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:25.444096 13005 server_base.cc:1034] running on GCE node
I20260926 07:39:25.444845 13005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:25.446138 13005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:25.447358 13005 hybrid_clock.cc:648] HybridClock initialized: now 1790408365447301 us; error 39 us; skew 500 ppm
I20260926 07:39:25.447674 13005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:25 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:25.450690 13005 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:25.452085 13005 webserver.cc:460] Webserver started at http://127.11.9.126:38561/ using document root <none> and password file <none>
I20260926 07:39:25.452732 13005 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:25.452847 13005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:25.453153 13005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:25.456045 13005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "be62022654b643548957295598c54e7e"
format_stamp: "Formatted at 2026-09-26 07:39:25 on dist-test-slave-x492"
server_key: "3a77271ad28fa74b2441c139ad4bfec0"
server_key_iv: "068fcb4814af59eb0e58632f7e747ae9"
server_key_version: "encryptionkey@0"
I20260926 07:39:25.456790 13005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "be62022654b643548957295598c54e7e"
format_stamp: "Formatted at 2026-09-26 07:39:25 on dist-test-slave-x492"
server_key: "3a77271ad28fa74b2441c139ad4bfec0"
server_key_iv: "068fcb4814af59eb0e58632f7e747ae9"
server_key_version: "encryptionkey@0"
I20260926 07:39:25.461670 13005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 07:39:25.464872 13020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:25.466207 13005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:25.466388 13005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "be62022654b643548957295598c54e7e"
format_stamp: "Formatted at 2026-09-26 07:39:25 on dist-test-slave-x492"
server_key: "3a77271ad28fa74b2441c139ad4bfec0"
server_key_iv: "068fcb4814af59eb0e58632f7e747ae9"
server_key_version: "encryptionkey@0"
I20260926 07:39:25.466599 13005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:25.479967 13005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:25.483608 13005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:25.483868 13005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:25.504324 13005 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42791
I20260926 07:39:25.504343 13081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42791 every 8 connection(s)
I20260926 07:39:25.505761 13005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:25.509413 13082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:25.516093 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13005
I20260926 07:39:25.516294 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:25.517144 11301 external_mini_cluster.cc:1428] Setting key 105d0d30f8a58d610e6beb138761d4ea
I20260926 07:39:25.526705 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Bootstrap starting.
I20260926 07:39:25.530827 13082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:25.532177 13082 log.cc:824] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:25.535642 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: No bootstrap required, opened a new log
Sep 26 07:39:25 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:25.547137 13082 raft_consensus.cc:348] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } }
I20260926 07:39:25.547575 13082 raft_consensus.cc:374] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:25.547680 13082 raft_consensus.cc:729] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be62022654b643548957295598c54e7e, State: Initialized, Role: FOLLOWER
I20260926 07:39:25.548359 13082 consensus_queue.cc:260] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } }
I20260926 07:39:25.548638 13082 raft_consensus.cc:388] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:25.548806 13082 raft_consensus.cc:482] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:25.548956 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:25.519282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.11.9.126:42791)
0926 07:39:25.520973 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:25.520985 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:25.521025 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:39:25.524266 (+ 3241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:25.524295 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:25.524963 (+ 668us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:25.527487 (+ 2524us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:25.527989 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:25.530027 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:25.532949 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:25.534154 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:25.534353 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:25.539366 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:25.539379 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:25.539398 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:25.539440 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:25.543413 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:25.544277 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:25.544286 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:25.544294 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:25.544514 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:25.544964 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:25.544968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:25.544972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:25.545693 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:25.545988 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:25.547191 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0926 07:39:25.547608 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":1218,"threads_started":1}
I20260926 07:39:25.549008 13082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:25.551703 13082 raft_consensus.cc:504] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } }
I20260926 07:39:25.552275 13082 leader_election.cc:302] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be62022654b643548957295598c54e7e; no voters:
I20260926 07:39:25.552881 13082 leader_election.cc:288] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:25.553082 13087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:25.553862 13087 raft_consensus.cc:686] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [term 1 LEADER]: Becoming Leader. State: Replica: be62022654b643548957295598c54e7e, State: Running, Role: LEADER
I20260926 07:39:25.554505 13087 consensus_queue.cc:237] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } }
I20260926 07:39:25.555042 13082 sys_catalog.cc:564] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:25.557578 13088 sys_catalog.cc:455] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be62022654b643548957295598c54e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } } }
I20260926 07:39:25.558014 13088 sys_catalog.cc:458] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:25.561056 13089 sys_catalog.cc:455] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader be62022654b643548957295598c54e7e. Latest consensus state: current_term: 1 leader_uuid: "be62022654b643548957295598c54e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be62022654b643548957295598c54e7e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42791 } } }
I20260926 07:39:25.561491 13089 sys_catalog.cc:458] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:25.562141 13096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:25.567348 13096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:25.576239 13096 catalog_manager.cc:1347] Generated new cluster ID: 062b7a6973104e5389f787ad8cb12ebb
I20260926 07:39:25.576359 13096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:25.597383 13096 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:25.598654 13096 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:25.604760 13096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Generated new TSK 0
I20260926 07:39:25.605556 13096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:25.658735 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42791
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:25.999198 13108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:25.999502 13108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:25.999612 13108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:26.006649 13108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:26.006759 13108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:26.006844 13108 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 07:39:26.006896 13108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:26.006958 13108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:26.017019 13108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:26.018323 13108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:26.019569 13108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:26.032369 13113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:26.032332 13114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:26.032295 13116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:26.033550 13108 server_base.cc:1034] running on GCE node
I20260926 07:39:26.034157 13108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:26.035063 13108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:26.036302 13108 hybrid_clock.cc:648] HybridClock initialized: now 1790408366036260 us; error 50 us; skew 500 ppm
I20260926 07:39:26.036610 13108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:26 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:26.039620 13108 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:26.041054 13108 webserver.cc:460] Webserver started at http://127.11.9.65:36417/ using document root <none> and password file <none>
I20260926 07:39:26.041719 13108 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:26.041870 13108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:26.042165 13108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:26.044874 13108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "d667c5c017f3771fb0887755dc591239"
server_key_iv: "12d4c68865243b83d5c20f7bdf01faa5"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.045617 13108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "d667c5c017f3771fb0887755dc591239"
server_key_iv: "12d4c68865243b83d5c20f7bdf01faa5"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.050555 13108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:26.053870 13123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:26.055186 13108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:26.055382 13108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "d667c5c017f3771fb0887755dc591239"
server_key_iv: "12d4c68865243b83d5c20f7bdf01faa5"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.055584 13108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:26.082180 13108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:26.086113 13108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:26.086403 13108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:26.088920 13108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:26.089046 13108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:26.089236 13108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:26.089349 13108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 26 07:39:26 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:26.111655 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.089815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:56361 (local address 127.11.9.126:42791)
0926 07:39:26.090399 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:26.090407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:26.091613 (+ 1206us) server_negotiation.cc:407] Connection header received
0926 07:39:26.093666 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:26.093672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:26.093771 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:26.093889 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:26.096601 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.097520 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.099897 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.100493 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.100652 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.104943 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:26.104955 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:26.104963 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:26.105018 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:26.108347 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.109377 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.109386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.109389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.109521 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.109947 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.109952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.109955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.110263 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:26.110411 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:26.111065 (+ 654us) server_negotiation.cc:299] Negotiation successful
0926 07:39:26.111280 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":338,"threads_started":1}
I20260926 07:39:26.112676 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.090508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.65:56361)
0926 07:39:26.091413 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:26.091470 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:26.093388 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:26.094319 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:26.094332 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:26.095249 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:26.096371 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.096417 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.097719 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.097722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.099746 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.099758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.100982 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.100987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.101239 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.102099 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:26.102140 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:26.104717 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:26.108510 (+ 3793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.108521 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.108535 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.109238 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.109630 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.109636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.109639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.109829 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.110389 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:26.110397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:26.110724 (+ 327us) client_negotiation.cc:769] Sending connection context
0926 07:39:26.110983 (+ 259us) client_negotiation.cc:240] Negotiation successful
0926 07:39:26.111461 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":391,"threads_started":1}
I20260926 07:39:26.134310 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.119908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:53075 (local address 127.11.9.126:42791)
0926 07:39:26.120089 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:26.120096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:26.120311 (+ 215us) server_negotiation.cc:407] Connection header received
0926 07:39:26.120534 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:26.120539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:26.120607 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:26.120702 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:26.121698 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.122612 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.124679 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.125331 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.125490 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.128061 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:26.128065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:26.128068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:26.128096 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:26.131223 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.132044 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.132047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.132049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.132167 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.132621 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.132628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.132633 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.133007 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:26.133212 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:26.133718 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 07:39:26.133928 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:39:26.134377 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.119418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.65:53075)
0926 07:39:26.120219 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:26.120233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:26.120395 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:26.120723 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:26.120727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:26.121225 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:26.121560 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.121570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.122790 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.122796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.124505 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.124521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.125552 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.125558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.125803 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.126489 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:26.126520 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:26.127871 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:26.131343 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.131347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.131352 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.131930 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.132284 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.132288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.132292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.132477 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.133207 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:26.133214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:26.133456 (+ 242us) client_negotiation.cc:769] Sending connection context
0926 07:39:26.133628 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 07:39:26.133943 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":503,"threads_started":1}
I20260926 07:39:26.142979 13108 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44983
I20260926 07:39:26.143025 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44983 every 8 connection(s)
I20260926 07:39:26.144162 13108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:26.146188 13244 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42791
I20260926 07:39:26.146404 13244 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:26.147104 13244 heartbeater.cc:511] Master 127.11.9.126:42791 requested a full tablet report, sending...
I20260926 07:39:26.148937 13037 ts_manager.cc:194] Registered new tserver with Master: a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983)
I20260926 07:39:26.150401 13037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:56361
I20260926 07:39:26.151201 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13108
I20260926 07:39:26.151422 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:26.152067 11301 external_mini_cluster.cc:1428] Setting key fc4defea3dd95d359aa25d7ff6733813
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:26.190433 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42791
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:26.501875 13248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:26.502133 13248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:26.502341 13248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:26.509447 13248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:26.509579 13248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:26.509727 13248 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 07:39:26.509824 13248 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:26.509950 13248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:26.519459 13248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:26.520639 13248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:26.521971 13248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:26.534626 13256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:26.534659 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:26.534560 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:26.536217 13248 server_base.cc:1034] running on GCE node
I20260926 07:39:26.536928 13248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:26.538012 13248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:26.539253 13248 hybrid_clock.cc:648] HybridClock initialized: now 1790408366539199 us; error 50 us; skew 500 ppm
I20260926 07:39:26.539577 13248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:26 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:26.542618 13248 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:26.544080 13248 webserver.cc:460] Webserver started at http://127.11.9.66:37307/ using document root <none> and password file <none>
I20260926 07:39:26.544739 13248 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:26.544890 13248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:26.545249 13248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:26.547992 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "a4dce639332ba2103d3f0d29e86a9ff8"
server_key_iv: "3dd275dc467d8b992479246673869fdd"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.548707 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "a4dce639332ba2103d3f0d29e86a9ff8"
server_key_iv: "3dd275dc467d8b992479246673869fdd"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.553668 13248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 07:39:26.557029 13263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:26.558264 13248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:26.558424 13248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
format_stamp: "Formatted at 2026-09-26 07:39:26 on dist-test-slave-x492"
server_key: "a4dce639332ba2103d3f0d29e86a9ff8"
server_key_iv: "3dd275dc467d8b992479246673869fdd"
server_key_version: "encryptionkey@0"
I20260926 07:39:26.558617 13248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:26.572041 13248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:26.575685 13248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:26.575963 13248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:26.578594 13248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:26.578727 13248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:26.578857 13248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:26.578941 13248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:39:26 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:26.601001 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.579038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:35573 (local address 127.11.9.126:42791)
0926 07:39:26.579209 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:26.579217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:26.580475 (+ 1258us) server_negotiation.cc:407] Connection header received
0926 07:39:26.582537 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:26.582543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:26.582645 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:26.582755 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:26.585047 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.586396 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.588903 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.589417 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.589590 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.594628 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:26.594636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:26.594643 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:26.594690 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:26.598028 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.598786 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.598791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.598795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.598914 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.599354 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.599360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.599364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.599629 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:26.599770 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:26.600437 (+ 667us) server_negotiation.cc:299] Negotiation successful
0926 07:39:26.600642 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 07:39:26.602342 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.579589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.66:35573)
0926 07:39:26.580324 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:26.580365 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:26.582259 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:26.583026 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:26.583040 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:26.583891 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:26.584723 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.584741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.586585 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.586591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.588750 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.588762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.589690 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.589700 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.590043 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.590897 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:26.590949 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:26.594323 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:26.598161 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.598169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.598186 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.598616 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.599033 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.599038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.599042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.599240 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.599758 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:26.599765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:26.600089 (+ 324us) client_negotiation.cc:769] Sending connection context
0926 07:39:26.600349 (+ 260us) client_negotiation.cc:240] Negotiation successful
0926 07:39:26.600842 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":362,"threads_started":1}
I20260926 07:39:26.621759 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.607919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.66:58641)
0926 07:39:26.608819 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:26.608840 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:26.609058 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:26.609460 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:26.609465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:26.609880 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:26.610151 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.610159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.611111 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.611116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.612109 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:26.612119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.612860 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.612868 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:26.613029 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.613877 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:26.613901 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:26.614944 (+ 1043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:26.618699 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.618707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.618711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.619341 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.619712 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:26.619718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:26.619723 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:26.619891 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:26.620688 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:26.620691 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:26.620830 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 07:39:26.620936 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 07:39:26.621153 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":557,"threads_started":1}
I20260926 07:39:26.621834 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:26.607935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:58641 (local address 127.11.9.126:42791)
0926 07:39:26.608137 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:26.608144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:26.608933 (+ 789us) server_negotiation.cc:407] Connection header received
0926 07:39:26.609238 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:26.609244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:26.609327 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:26.609437 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:26.610285 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.610955 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.612269 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:26.612676 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:26.612800 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:26.615128 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:26.615137 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:26.615145 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:26.615187 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:26.618562 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.619464 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.619470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.619476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.619605 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:26.620029 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:26.620037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:26.620043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:26.620557 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:26.620744 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:26.621092 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 07:39:26.621398 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:39:26.632882 13248 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:46065
I20260926 07:39:26.632912 13382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:46065 every 8 connection(s)
I20260926 07:39:26.634091 13248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:26.636062 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13248
I20260926 07:39:26.636073 13383 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42791
I20260926 07:39:26.636224 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:26.636310 13383 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:26.636691 11301 external_mini_cluster.cc:1428] Setting key 8ef6cc131901883a17152703c240b5d2
I20260926 07:39:26.636996 13383 heartbeater.cc:511] Master 127.11.9.126:42791 requested a full tablet report, sending...
I20260926 07:39:26.638499 13036 ts_manager.cc:194] Registered new tserver with Master: 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065)
I20260926 07:39:26.639560 13036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:35573
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:26.676338 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42791
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:45779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:26.991449 13387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:26.991706 13387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:26.991807 13387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:26.998940 13387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:26.999045 13387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:26.999127 13387 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 07:39:26.999178 13387 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:39:26.999238 13387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:27.008711 13387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:27.009938 13387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:27.011272 13387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:27.024272 13392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:27.024250 13395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:27.024345 13393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:27.025111 13387 server_base.cc:1034] running on GCE node
I20260926 07:39:27.025695 13387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:27.026530 13387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:27.027719 13387 hybrid_clock.cc:648] HybridClock initialized: now 1790408367027663 us; error 50 us; skew 500 ppm
I20260926 07:39:27.028095 13387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:27 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:27.031198 13387 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:27.032667 13387 webserver.cc:460] Webserver started at http://127.11.9.67:41345/ using document root <none> and password file <none>
I20260926 07:39:27.033363 13387 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:27.033468 13387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:27.033723 13387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:27.036434 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
format_stamp: "Formatted at 2026-09-26 07:39:27 on dist-test-slave-x492"
server_key: "40940027a6db69bb3d72237e80a4b1ec"
server_key_iv: "fc3eb258a94135b0f078bfe5d12c531c"
server_key_version: "encryptionkey@0"
I20260926 07:39:27.037133 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
format_stamp: "Formatted at 2026-09-26 07:39:27 on dist-test-slave-x492"
server_key: "40940027a6db69bb3d72237e80a4b1ec"
server_key_iv: "fc3eb258a94135b0f078bfe5d12c531c"
server_key_version: "encryptionkey@0"
I20260926 07:39:27.042172 13387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:27.045475 13402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:27.046808 13387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:39:27.046972 13387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
format_stamp: "Formatted at 2026-09-26 07:39:27 on dist-test-slave-x492"
server_key: "40940027a6db69bb3d72237e80a4b1ec"
server_key_iv: "fc3eb258a94135b0f078bfe5d12c531c"
server_key_version: "encryptionkey@0"
I20260926 07:39:27.047165 13387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:27.073359 13387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:27.077397 13387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:27.077694 13387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:27.080181 13387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:27.080339 13387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:27.080478 13387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:27.080600 13387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:39:27 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:27.105933 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.081097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:52217 (local address 127.11.9.126:42791)
0926 07:39:27.081376 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.081382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.082778 (+ 1396us) server_negotiation.cc:407] Connection header received
0926 07:39:27.085398 (+ 2620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.085419 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.085529 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.085651 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:27.087832 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.088924 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.091341 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.092258 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.092447 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.097868 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:27.097877 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:27.097885 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:27.097934 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:27.101591 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.102666 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.102674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.102679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.102874 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.103460 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.103466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.103469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.103850 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:27.104022 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.105192 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.105468 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260926 07:39:27.107769 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.081662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.67:52217)
0926 07:39:27.082591 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.082637 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.084987 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.085888 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.085896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.086677 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:27.087628 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.087661 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.089086 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.089092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.091160 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.091174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.092943 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.092955 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.093318 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.094230 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:27.094254 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:27.097517 (+ 3263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:27.101734 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.101743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.101760 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.102456 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.102999 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.103008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.103010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.103331 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.104007 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:27.104018 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:27.104609 (+ 591us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.105001 (+ 392us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.105748 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":419,"threads_started":1}
I20260926 07:39:27.127190 13387 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:35837
I20260926 07:39:27.127235 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:35837 every 8 connection(s)
I20260926 07:39:27.128952 13387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:27.131858 13504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.115452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.11.9.67:52193)
0926 07:39:27.116152 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.116185 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.116354 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.116709 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.116714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.117481 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:27.117951 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.117961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.118943 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.118950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.120688 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.120704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.121747 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.121751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.121861 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.122527 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:27.122557 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:27.124277 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:27.128646 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.128652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.128657 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.129221 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.129862 (+ 641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.129869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.129872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.130049 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.130684 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:27.130689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:27.130930 (+ 241us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.131096 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.131411 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":369,"threads_started":1}
I20260926 07:39:27.132290 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.115444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:52193 (local address 127.11.9.126:42791)
0926 07:39:27.115675 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.115684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.116275 (+ 591us) server_negotiation.cc:407] Connection header received
0926 07:39:27.116486 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.116492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.116574 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.116681 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:27.118110 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.118799 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.120837 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.121392 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.121536 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.124735 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:27.124746 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:27.124754 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:27.124836 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:27.128509 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.129342 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.129347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.129351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.129503 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.130166 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.130192 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.130198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.130575 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:27.130728 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.131184 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.131392 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:39:27.132459 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13387
I20260926 07:39:27.132632 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:27.133267 11301 external_mini_cluster.cc:1428] Setting key 6abe2a0d8cf1439117580954aa8e9bc6
I20260926 07:39:27.133807 13522 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42791
I20260926 07:39:27.134059 13522 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:27.134881 13522 heartbeater.cc:511] Master 127.11.9.126:42791 requested a full tablet report, sending...
I20260926 07:39:27.136474 13037 ts_manager.cc:194] Registered new tserver with Master: dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:27.137584 13037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:52217
I20260926 07:39:27.148313 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:27.153306 13244 heartbeater.cc:503] Master 127.11.9.126:42791 was elected leader, sending a full tablet report...
Sep 26 07:39:27 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 07:39:27 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:27.197060 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.180145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.11.9.126:42791)
0926 07:39:27.180319 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.180324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.180700 (+ 376us) server_negotiation.cc:407] Connection header received
0926 07:39:27.180848 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.180852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.180913 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.181013 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:27.182382 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.183108 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.184237 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.184625 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.184715 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.192568 (+ 7853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:27.192573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:27.192576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:27.192608 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:27.194897 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.195461 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.195464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.195466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.195577 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.195875 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.195878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.195880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.196119 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:27.196277 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.196525 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.196673 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:39:27.200701 13037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:27.207381 13037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:27.231582 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.219732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.11.9.66:46065)
0926 07:39:27.221665 (+ 1933us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.221677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.222950 (+ 1273us) server_negotiation.cc:407] Connection header received
0926 07:39:27.223051 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.223059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.223318 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.223626 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.224817 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.226193 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.228863 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.230295 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.230440 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.230633 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.231126 (+ 493us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.231287 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1734,"thread_start_us":1577,"threads_started":1}
I20260926 07:39:27.231882 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.219227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.11.9.67:35837)
0926 07:39:27.220078 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.220090 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.222524 (+ 2434us) server_negotiation.cc:407] Connection header received
0926 07:39:27.222872 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.222883 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.223210 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.223408 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.225399 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.227036 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.229266 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.230421 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.230584 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.230923 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.231285 (+ 362us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.231440 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":707,"threads_started":1}
I20260926 07:39:27.232404 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.219163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46065 (local address 127.0.0.1:40484)
0926 07:39:27.222669 (+ 3506us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.222683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.222811 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.223888 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.223896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.223928 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.224621 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.224638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.226336 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.226341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.228650 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.228664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.230500 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.230508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.230835 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.230916 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.231070 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.231983 (+ 913us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3410,"mutex_wait_us":819,"thread_start_us":2670,"threads_started":1}
I20260926 07:39:27.231666 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.218939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35837 (local address 127.0.0.1:52512)
0926 07:39:27.222333 (+ 3394us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.222384 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.222611 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.223858 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.223868 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.223908 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.225143 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.225158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.227155 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.227163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.229063 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.229075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.230794 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.230801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.230967 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.231005 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.231103 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.231276 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3064,"mutex_wait_us":5,"thread_start_us":3475,"threads_started":1}
I20260926 07:39:27.234549 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.222815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.0.0.1:46086)
0926 07:39:27.225582 (+ 2767us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.225600 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.225714 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.226361 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.226366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.226394 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.226844 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.226854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.228683 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.228689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.230857 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.230868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.233756 (+ 2888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.233764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.233939 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.233970 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.234072 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.234192 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2655,"mutex_wait_us":1,"thread_start_us":2537,"threads_started":1}
I20260926 07:39:27.234565 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.223087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.11.9.65:44983)
0926 07:39:27.224446 (+ 1359us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.224452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.225828 (+ 1376us) server_negotiation.cc:407] Connection header received
0926 07:39:27.225922 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.225929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.226133 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.226306 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.227024 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.228507 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.231043 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.232552 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.232681 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.232964 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.234131 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.234278 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":1114,"threads_started":1}
I20260926 07:39:27.236295 13313 tablet_service.cc:1467] Processing CreateTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc (DEFAULT_TABLE table=test-table [id=9833a5fd18974bce882af13ec3be94ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.236472 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc (DEFAULT_TABLE table=test-table [id=9833a5fd18974bce882af13ec3be94ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.237757 13313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4672b8ba493e4acf94a712c9dcb861dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.237790 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4672b8ba493e4acf94a712c9dcb861dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.237845 13174 tablet_service.cc:1467] Processing CreateTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc (DEFAULT_TABLE table=test-table [id=9833a5fd18974bce882af13ec3be94ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.239116 13174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4672b8ba493e4acf94a712c9dcb861dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.248878 13541 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Bootstrap starting.
I20260926 07:39:27.252307 13541 tablet_bootstrap.cc:654] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.253374 13541 log.cc:824] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:27.253937 13542 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Bootstrap starting.
I20260926 07:39:27.254789 13543 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Bootstrap starting.
I20260926 07:39:27.257957 13543 tablet_bootstrap.cc:654] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.258562 13541 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: No bootstrap required, opened a new log
I20260926 07:39:27.258598 13542 tablet_bootstrap.cc:654] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.258869 13541 ts_tablet_manager.cc:1397] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260926 07:39:27.259057 13543 log.cc:824] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:27.259842 13542 log.cc:824] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:27.261725 13543 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: No bootstrap required, opened a new log
I20260926 07:39:27.261971 13543 ts_tablet_manager.cc:1397] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 07:39:27.267553 13542 tablet_bootstrap.cc:492] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: No bootstrap required, opened a new log
I20260926 07:39:27.267822 13542 ts_tablet_manager.cc:1397] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260926 07:39:27.269577 13543 raft_consensus.cc:348] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.269977 13543 raft_consensus.cc:374] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.270083 13543 raft_consensus.cc:729] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.270326 13541 raft_consensus.cc:348] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.270720 13541 raft_consensus.cc:374] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.270807 13541 raft_consensus.cc:729] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.270711 13543 consensus_queue.cc:260] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.271360 13541 consensus_queue.cc:260] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.272620 13522 heartbeater.cc:503] Master 127.11.9.126:42791 was elected leader, sending a full tablet report...
I20260926 07:39:27.273080 13541 ts_tablet_manager.cc:1428] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 07:39:27.278350 13543 ts_tablet_manager.cc:1428] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260926 07:39:27.279269 13542 raft_consensus.cc:348] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.279592 13542 raft_consensus.cc:374] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.279691 13542 raft_consensus.cc:729] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eacecc48b6d4baca02bd2b8d76586e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.280784 13542 consensus_queue.cc:260] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.281565 13548 raft_consensus.cc:482] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:27.281841 13383 heartbeater.cc:503] Master 127.11.9.126:42791 was elected leader, sending a full tablet report...
I20260926 07:39:27.281805 13548 raft_consensus.cc:504] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.282373 13542 ts_tablet_manager.cc:1428] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 07:39:27.283458 13548 leader_election.cc:288] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983)
I20260926 07:39:27.291075 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.283658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46065 (local address 127.11.9.67:44573)
0926 07:39:27.284909 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.284933 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.285070 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.286223 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.286228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.286275 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.286432 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.286440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.287455 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.287458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.289214 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.289228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.290561 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.290566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.290688 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.290711 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.290780 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.290866 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":1}
I20260926 07:39:27.291200 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.285748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:44573 (local address 127.11.9.66:46065)
0926 07:39:27.285964 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.285972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.285991 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:39:27.286049 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.286056 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.286121 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.286208 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.286564 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.287349 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.289450 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.290411 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.290540 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.290639 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.290833 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.290958 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 07:39:27.291973 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4672b8ba493e4acf94a712c9dcb861dc" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" is_pre_election: true
I20260926 07:39:27.292263 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.285396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:38017 (local address 127.11.9.65:44983)
0926 07:39:27.285562 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.285566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.286123 (+ 557us) server_negotiation.cc:407] Connection header received
0926 07:39:27.286321 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.286326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.286402 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.286461 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.287213 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.287968 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.290002 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.291174 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.291313 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.291422 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.291912 (+ 490us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.292037 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:39:27.292323 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.283829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.11.9.67:38017)
0926 07:39:27.286035 (+ 2206us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.286055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.286175 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.286646 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.286650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.286675 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.287037 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.287049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.288101 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.288108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.289847 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.289856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.291577 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.291583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.291753 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.291776 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.291881 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.291997 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":1054,"threads_started":1}
I20260926 07:39:27.292485 13338 raft_consensus.cc:2454] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 0.
I20260926 07:39:27.293200 13404 leader_election.cc:302] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, dc5873b6c8564cd5b2fe939718f81bb8; no voters:
I20260926 07:39:27.293336 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4672b8ba493e4acf94a712c9dcb861dc" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" is_pre_election: true
I20260926 07:39:27.293623 13548 raft_consensus.cc:2781] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:27.293771 13548 raft_consensus.cc:482] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:27.293887 13548 raft_consensus.cc:3037] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.293808 13199 raft_consensus.cc:2454] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 0.
I20260926 07:39:27.296119 13548 raft_consensus.cc:504] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.296716 13548 leader_election.cc:288] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 election: Requested vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983)
I20260926 07:39:27.297071 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4672b8ba493e4acf94a712c9dcb861dc" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
I20260926 07:39:27.297075 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4672b8ba493e4acf94a712c9dcb861dc" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
I20260926 07:39:27.297271 13199 raft_consensus.cc:3037] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.297315 13338 raft_consensus.cc:3037] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.299307 13199 raft_consensus.cc:2454] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 1.
I20260926 07:39:27.299415 13338 raft_consensus.cc:2454] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 1.
I20260926 07:39:27.299690 13403 leader_election.cc:302] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ca52f612ac43b88ce1f953b2b6cffc, dc5873b6c8564cd5b2fe939718f81bb8; no voters:
I20260926 07:39:27.299965 13548 raft_consensus.cc:2781] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:27.300312 13548 raft_consensus.cc:686] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 LEADER]: Becoming Leader. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Running, Role: LEADER
I20260926 07:39:27.300753 13548 consensus_queue.cc:237] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } }
I20260926 07:39:27.305074 13036 catalog_manager.cc:5495] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } health_report { overall_health: UNKNOWN } } }
Sep 26 07:39:27 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:27.365056 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.351613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.11.9.126:42791)
0926 07:39:27.351808 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.351819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.352423 (+ 604us) server_negotiation.cc:407] Connection header received
0926 07:39:27.352569 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.352573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.352630 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.352712 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:27.353953 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.354695 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.356121 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.356513 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.356615 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.359882 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:27.359887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:27.359890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:27.359926 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:27.362454 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.363152 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.363160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.363163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.363273 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:27.363633 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:27.363638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:27.363641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:27.363877 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:27.364027 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.364418 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.364714 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":63}
I20260926 07:39:27.365087 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.351640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42791 (local address 127.0.0.1:48146)
0926 07:39:27.352309 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.352325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.352451 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.352755 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.352758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.353527 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:27.353823 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.353832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.354836 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.354842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.355996 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.356007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.356664 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.356668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.356798 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.357509 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:27.357525 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:27.359710 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:27.362594 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.362604 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.362611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.363042 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.363386 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:27.363391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:27.363394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:27.363521 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:27.364004 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:27.364012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:27.364161 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.364321 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.364598 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":363,"threads_started":1}
I20260926 07:39:27.369670 13036 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:48146:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 07:39:27.370636 13036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:27.376955 13313 tablet_service.cc:1467] Processing CreateTablet for tablet c81f0ba154dd49e9ba94f58f8b5fb321 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d783fc5462b6407bb4c9696b2765ce52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:27.376818 13174 tablet_service.cc:1467] Processing CreateTablet for tablet c81f0ba154dd49e9ba94f58f8b5fb321 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d783fc5462b6407bb4c9696b2765ce52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:27.377833 13313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81f0ba154dd49e9ba94f58f8b5fb321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.377843 13174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81f0ba154dd49e9ba94f58f8b5fb321. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:39:27.382984 13523 tablet.cc:2369] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:27.383365 13542 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: Bootstrap starting.
I20260926 07:39:27.383800 13452 tablet_service.cc:1467] Processing CreateTablet for tablet c81f0ba154dd49e9ba94f58f8b5fb321 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d783fc5462b6407bb4c9696b2765ce52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:39:27.384500 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81f0ba154dd49e9ba94f58f8b5fb321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.384804 13542 tablet_bootstrap.cc:654] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.386003 13543 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: Bootstrap starting.
I20260926 07:39:27.387285 13543 tablet_bootstrap.cc:654] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.388595 13542 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: No bootstrap required, opened a new log
I20260926 07:39:27.388746 13542 ts_tablet_manager.cc:1397] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
W20260926 07:39:27.388938 13384 tablet.cc:2369] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:27.390091 13541 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: Bootstrap starting.
I20260926 07:39:27.389999 13542 raft_consensus.cc:348] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.390259 13542 raft_consensus.cc:374] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.390365 13542 raft_consensus.cc:729] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eacecc48b6d4baca02bd2b8d76586e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.390527 13543 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: No bootstrap required, opened a new log
I20260926 07:39:27.390654 13543 ts_tablet_manager.cc:1397] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 07:39:27.390597 13542 consensus_queue.cc:260] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.391191 13542 ts_tablet_manager.cc:1428] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:27.391078 13549 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.391466 13549 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.391386 13543 raft_consensus.cc:348] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.391649 13543 raft_consensus.cc:374] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.391742 13541 tablet_bootstrap.cc:654] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.391741 13543 raft_consensus.cc:729] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.391968 13543 consensus_queue.cc:260] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.392509 13547 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.392879 13547 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.393342 13543 ts_tablet_manager.cc:1428] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:27.394189 13541 tablet_bootstrap.cc:492] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: No bootstrap required, opened a new log
I20260926 07:39:27.394378 13541 ts_tablet_manager.cc:1397] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 07:39:27.395110 13541 raft_consensus.cc:348] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.395401 13541 raft_consensus.cc:374] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.395531 13541 raft_consensus.cc:729] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.395750 13541 consensus_queue.cc:260] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.396176 13548 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.396467 13548 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 07:39:27.396466 13245 tablet.cc:2369] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:27.397038 13541 ts_tablet_manager.cc:1428] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:27.541607 13547 raft_consensus.cc:482] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:27.541832 13547 raft_consensus.cc:504] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.543388 13547 leader_election.cc:288] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:27.550117 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.543795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:44517 (local address 127.11.9.67:35837)
0926 07:39:27.544026 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.544035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.544755 (+ 720us) server_negotiation.cc:407] Connection header received
0926 07:39:27.544995 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.544999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.545075 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.545188 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.545807 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.546508 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.548399 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.549116 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.549314 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.549443 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.549725 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.549856 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 07:39:27.550171 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.543661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38769 (local address 127.11.9.66:46065)
0926 07:39:27.543865 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.543892 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.544690 (+ 798us) server_negotiation.cc:407] Connection header received
0926 07:39:27.544900 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.544909 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.545012 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.545082 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:27.545848 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.546619 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.548399 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.549153 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.549324 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.549419 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.549793 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.549914 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:39:27.550292 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.543678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35837 (local address 127.11.9.65:44517)
0926 07:39:27.544611 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.544629 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.544804 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.545234 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.545242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.545271 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.545651 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.545661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.546651 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.546659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.548238 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.548249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.549374 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.549381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.549558 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.549595 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.549667 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.549806 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":70,"thread_start_us":625,"threads_started":1}
I20260926 07:39:27.550420 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.543557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46065 (local address 127.11.9.65:38769)
0926 07:39:27.544544 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.544567 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.544750 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.545127 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.545134 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.545187 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:27.545663 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.545672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.546736 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.546740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.548236 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.548247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.549415 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.549422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.549608 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.549630 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.549745 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.549861 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":39,"spinlock_wait_cycles":22400,"thread_start_us":643,"threads_started":1}
I20260926 07:39:27.551008 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81f0ba154dd49e9ba94f58f8b5fb321" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" is_pre_election: true
I20260926 07:39:27.551265 13338 raft_consensus.cc:2454] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:27.551281 13472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81f0ba154dd49e9ba94f58f8b5fb321" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" is_pre_election: true
I20260926 07:39:27.551697 13472 raft_consensus.cc:2454] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:27.551782 13125 leader_election.cc:302] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:27.552210 13547 raft_consensus.cc:2781] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:27.552350 13547 raft_consensus.cc:482] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:27.552448 13547 raft_consensus.cc:3037] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.554560 13547 raft_consensus.cc:504] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.555217 13547 leader_election.cc:288] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Requested vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:27.555619 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81f0ba154dd49e9ba94f58f8b5fb321" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
I20260926 07:39:27.555711 13472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81f0ba154dd49e9ba94f58f8b5fb321" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
I20260926 07:39:27.555840 13338 raft_consensus.cc:3037] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.555923 13472 raft_consensus.cc:3037] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.558796 13338 raft_consensus.cc:2454] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:27.558965 13472 raft_consensus.cc:2454] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:27.559245 13125 leader_election.cc:302] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:27.559600 13547 raft_consensus.cc:2781] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:27.560040 13547 raft_consensus.cc:686] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 LEADER]: Becoming Leader. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Running, Role: LEADER
I20260926 07:39:27.560542 13547 consensus_queue.cc:237] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.564046 13564 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: TxnStatusTablet state changed. Reason: New leader a6ca52f612ac43b88ce1f953b2b6cffc. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.564461 13564 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:27.565146 13566 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 07:39:27.565448 13566 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 07:39:27.565447 13036 catalog_manager.cc:5495] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65). New cstate: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:27.627454 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.621218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.11.9.65:44983)
0926 07:39:27.621405 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.621411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.621524 (+ 113us) server_negotiation.cc:407] Connection header received
0926 07:39:27.621792 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.621795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.621883 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.621998 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.622394 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.623143 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.625027 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.625462 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.625565 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.626117 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.626818 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.626925 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.627075 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.627136 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:39:27.627566 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.621158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.0.0.1:46100)
0926 07:39:27.621404 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.621433 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.621602 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.622041 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.622046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.622059 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:27.622253 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.622264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.623263 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.623268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.624898 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.624912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.625613 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.625619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.625867 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.625938 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.626929 (+ 991us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.626951 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.627042 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.627189 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 07:39:27.635351 13472 raft_consensus.cc:1261] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.635383 13338 raft_consensus.cc:1261] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.636329 13547 consensus_queue.cc:1035] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:27.636694 13564 consensus_queue.cc:1035] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:27.658354 13548 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: TxnStatusTablet state changed. Reason: New leader a6ca52f612ac43b88ce1f953b2b6cffc. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.658756 13548 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.662339 13549 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: TxnStatusTablet state changed. Reason: New leader a6ca52f612ac43b88ce1f953b2b6cffc. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.662638 13549 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.662940 13569 mvcc.cc:205] Tried to move back new op lower bound from 7333512673824358400 to 7333512673531916288. Current Snapshot: MvccSnapshot[applied={T|T < 7333512673824358400}]
I20260926 07:39:27.665126 13570 mvcc.cc:205] Tried to move back new op lower bound from 7333512673824358400 to 7333512673531916288. Current Snapshot: MvccSnapshot[applied={T|T < 7333512673824358400}]
I20260926 07:39:27.667725 13549 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.668004 13549 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P 6eacecc48b6d4baca02bd2b8d76586e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.668722 13548 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.669008 13548 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P dc5873b6c8564cd5b2fe939718f81bb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:39:27.673792 13577 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.674294 13577 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:27.674320 13547 tablet_replica.cc:439] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } } }
I20260926 07:39:27.674723 13547 tablet_replica.cc:442] T c81f0ba154dd49e9ba94f58f8b5fb321 P a6ca52f612ac43b88ce1f953b2b6cffc: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:39:27.685140 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.679721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.11.9.67:35837)
0926 07:39:27.679906 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.679913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.680157 (+ 244us) server_negotiation.cc:407] Connection header received
0926 07:39:27.680388 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.680391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.680477 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.680626 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.680968 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.681797 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.683073 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.683463 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.683600 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.684065 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.684563 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.684713 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.684791 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.684865 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:39:27.700981 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.695131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.11.9.67:54431)
0926 07:39:27.695825 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.695840 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.695944 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.696269 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.696272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.696281 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:27.696534 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.696541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.697984 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.697988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.699022 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.699033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.700005 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.700009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.700113 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.700157 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.700566 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.700573 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.700630 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.700727 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":370,"threads_started":1}
I20260926 07:39:27.701035 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.695183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:54431 (local address 127.11.9.65:44983)
0926 07:39:27.695370 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.695378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.695916 (+ 538us) server_negotiation.cc:407] Connection header received
0926 07:39:27.696058 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.696063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.696157 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.696253 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.696684 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.697872 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.699153 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.699841 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.699961 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.700328 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.700471 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.700560 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.700676 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.700749 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:39:27.709630 13337 raft_consensus.cc:1261] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Refusing update from remote peer dc5873b6c8564cd5b2fe939718f81bb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.709924 13199 raft_consensus.cc:1261] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Refusing update from remote peer dc5873b6c8564cd5b2fe939718f81bb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.710260 13551 consensus_queue.cc:1035] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:27.710618 13548 consensus_queue.cc:1035] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:27.748118 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.741413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:56903 (local address 127.11.9.67:35837)
0926 07:39:27.741592 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.741598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.741822 (+ 224us) server_negotiation.cc:407] Connection header received
0926 07:39:27.742040 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.742045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.742118 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.742205 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.742856 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.743745 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.745942 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.746554 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.746700 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.747361 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.747486 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.747568 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.747764 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.747841 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 07:39:27.748236 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.740935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35837 (local address 127.11.9.65:56903)
0926 07:39:27.741720 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.741736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.741904 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.742230 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.742235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.742246 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:27.742702 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.742716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.743916 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.743921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.745810 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.745825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.746910 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.746919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.747122 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.747178 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.747620 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.747630 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.747736 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.747877 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":399,"threads_started":1}
I20260926 07:39:27.784446 13036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:27.784734 13036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:27.788575 13036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Sending DeleteTablet for 3 replicas of tablet 4672b8ba493e4acf94a712c9dcb861dc
I20260926 07:39:27.789737 13452 tablet_service.cc:1514] Processing DeleteTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:27 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52512
I20260926 07:39:27.789917 13313 tablet_service.cc:1514] Processing DeleteTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:27 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:40484
I20260926 07:39:27.791028 13174 tablet_service.cc:1514] Processing DeleteTablet for tablet 4672b8ba493e4acf94a712c9dcb861dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:27 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:46086
I20260926 07:39:27.795172 13036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:27.795837 13036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:27.801268 13593 tablet_replica.cc:330] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: stopping tablet replica
I20260926 07:39:27.801834 13593 raft_consensus.cc:2229] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:27.802258 13593 raft_consensus.cc:2258] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:27.804087 13593 ts_tablet_manager.cc:1905] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:27.804625 13592 tablet_replica.cc:330] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: stopping tablet replica
I20260926 07:39:27.805025 13592 raft_consensus.cc:2229] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:27.805624 13592 raft_consensus.cc:2258] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:27.809139 13313 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb60636f27466eb0687d8b86469b4f (DEFAULT_TABLE table=test-table [id=cc813ed476414935a7b814e7c3286468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.809317 13173 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb60636f27466eb0687d8b86469b4f (DEFAULT_TABLE table=test-table [id=cc813ed476414935a7b814e7c3286468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.809691 13313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb60636f27466eb0687d8b86469b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.809875 13173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb60636f27466eb0687d8b86469b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.816928 13543 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Bootstrap starting.
I20260926 07:39:27.818694 13543 tablet_bootstrap.cc:654] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.820431 13542 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Bootstrap starting.
I20260926 07:39:27.822155 13542 tablet_bootstrap.cc:654] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.808408 13451 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb60636f27466eb0687d8b86469b4f (DEFAULT_TABLE table=test-table [id=cc813ed476414935a7b814e7c3286468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:27.827378 13451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb60636f27466eb0687d8b86469b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:27.828480 13594 tablet_replica.cc:330] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: stopping tablet replica
I20260926 07:39:27.828945 13594 raft_consensus.cc:2229] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:27.829337 13594 raft_consensus.cc:2258] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:27.831615 13594 ts_tablet_manager.cc:1905] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:27.833559 13543 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: No bootstrap required, opened a new log
I20260926 07:39:27.833734 13543 ts_tablet_manager.cc:1397] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.002s
I20260926 07:39:27.834542 13543 raft_consensus.cc:348] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.834798 13543 raft_consensus.cc:374] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.834887 13543 raft_consensus.cc:729] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.835168 13543 consensus_queue.cc:260] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.835803 13543 ts_tablet_manager.cc:1428] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:27.838341 13592 ts_tablet_manager.cc:1905] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:27.840226 13594 ts_tablet_manager.cc:1918] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:27.840447 13594 log.cc:1196] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/4672b8ba493e4acf94a712c9dcb861dc
I20260926 07:39:27.841017 13594 ts_tablet_manager.cc:1939] T 4672b8ba493e4acf94a712c9dcb861dc P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting consensus metadata
I20260926 07:39:27.842381 13541 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Bootstrap starting.
I20260926 07:39:27.843777 13021 catalog_manager.cc:4841] TS a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983): tablet 4672b8ba493e4acf94a712c9dcb861dc (table test-table [id=9833a5fd18974bce882af13ec3be94ab]) successfully deleted
I20260926 07:39:27.844022 13541 tablet_bootstrap.cc:654] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:27.846405 13541 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: No bootstrap required, opened a new log
I20260926 07:39:27.846580 13541 ts_tablet_manager.cc:1397] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 07:39:27.847385 13541 raft_consensus.cc:348] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.847653 13541 raft_consensus.cc:374] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.847747 13541 raft_consensus.cc:729] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.848033 13541 consensus_queue.cc:260] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.848811 13542 tablet_bootstrap.cc:492] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: No bootstrap required, opened a new log
I20260926 07:39:27.848990 13542 ts_tablet_manager.cc:1397] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.003s
I20260926 07:39:27.849629 13541 ts_tablet_manager.cc:1428] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:27.850776 13542 raft_consensus.cc:348] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.851032 13542 raft_consensus.cc:374] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:27.851128 13542 raft_consensus.cc:729] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eacecc48b6d4baca02bd2b8d76586e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:27.852178 13542 consensus_queue.cc:260] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.852799 13542 ts_tablet_manager.cc:1428] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 07:39:27.853082 13592 ts_tablet_manager.cc:1918] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:27.853291 13592 log.cc:1196] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/4672b8ba493e4acf94a712c9dcb861dc
I20260926 07:39:27.854015 13593 ts_tablet_manager.cc:1918] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:27.854274 13593 log.cc:1196] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/4672b8ba493e4acf94a712c9dcb861dc
I20260926 07:39:27.854887 13593 ts_tablet_manager.cc:1939] T 4672b8ba493e4acf94a712c9dcb861dc P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting consensus metadata
I20260926 07:39:27.856706 13022 catalog_manager.cc:4841] TS 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065): tablet 4672b8ba493e4acf94a712c9dcb861dc (table test-table [id=9833a5fd18974bce882af13ec3be94ab]) successfully deleted
I20260926 07:39:27.857084 13592 ts_tablet_manager.cc:1939] T 4672b8ba493e4acf94a712c9dcb861dc P dc5873b6c8564cd5b2fe939718f81bb8: Deleting consensus metadata
I20260926 07:39:27.858479 13021 catalog_manager.cc:4841] TS dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837): tablet 4672b8ba493e4acf94a712c9dcb861dc (table test-table [id=9833a5fd18974bce882af13ec3be94ab]) successfully deleted
I20260926 07:39:27.885263 13564 raft_consensus.cc:482] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:27.885507 13564 raft_consensus.cc:504] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.886240 13564 leader_election.cc:288] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:27.887101 13337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb60636f27466eb0687d8b86469b4f" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" is_pre_election: true
I20260926 07:39:27.887394 13337 raft_consensus.cc:2454] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:27.887357 13472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb60636f27466eb0687d8b86469b4f" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" is_pre_election: true
I20260926 07:39:27.887603 13472 raft_consensus.cc:2454] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:27.887879 13125 leader_election.cc:302] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:27.888247 13564 raft_consensus.cc:2781] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:27.888351 13564 raft_consensus.cc:482] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:27.888458 13564 raft_consensus.cc:3037] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.890950 13564 raft_consensus.cc:504] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.891491 13564 leader_election.cc:288] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Requested vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:27.891947 13337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb60636f27466eb0687d8b86469b4f" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
I20260926 07:39:27.892179 13337 raft_consensus.cc:3037] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.892124 13472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb60636f27466eb0687d8b86469b4f" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
I20260926 07:39:27.892343 13472 raft_consensus.cc:3037] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:27.895164 13337 raft_consensus.cc:2454] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:27.895361 13472 raft_consensus.cc:2454] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:27.895607 13125 leader_election.cc:302] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:27.896064 13564 raft_consensus.cc:2781] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:27.896261 13564 raft_consensus.cc:686] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 LEADER]: Becoming Leader. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Running, Role: LEADER
I20260926 07:39:27.896550 13564 consensus_queue.cc:237] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:27.899636 13037 catalog_manager.cc:5495] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65). New cstate: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:27.942273 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.937148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.11.9.65:44983)
0926 07:39:27.937350 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.937356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.937375 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:39:27.937600 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.937606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.937681 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.937775 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.938150 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.938928 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.940391 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.940866 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.940999 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.941506 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.941624 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.941724 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.941855 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.941931 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 07:39:27.955267 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.950397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38931 (local address 127.11.9.65:44983)
0926 07:39:27.950587 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:27.950594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:27.950845 (+ 251us) server_negotiation.cc:407] Connection header received
0926 07:39:27.950881 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:27.950887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:27.950937 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:27.951048 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:27.951403 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.952252 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.953633 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.954058 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.954190 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.954548 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.954654 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.954753 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:27.954885 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 07:39:27.954951 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 07:39:27.955267 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:27.950362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.11.9.65:38931)
0926 07:39:27.950583 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:27.950606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:27.950745 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:27.951060 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:27.951064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:27.951075 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:39:27.951282 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.951291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.952387 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.952393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.953503 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:27.953515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:27.954225 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:27.954231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:27.954355 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:27.954393 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:39:27.954752 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:39:27.954764 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 07:39:27.954868 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 07:39:27.954996 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":24}
I20260926 07:39:27.966508 13337 raft_consensus.cc:1261] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.966498 13472 raft_consensus.cc:1261] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:27.967057 13577 consensus_queue.cc:1035] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:27.967577 13564 consensus_queue.cc:1035] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:39:28.002552 13569 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 07:39:29.594058 13035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:29.594273 13035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:29.597990 13035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Sending DeleteTablet for 3 replicas of tablet f9fb60636f27466eb0687d8b86469b4f
I20260926 07:39:29.598618 13174 tablet_service.cc:1514] Processing DeleteTablet for tablet f9fb60636f27466eb0687d8b86469b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:46086
I20260926 07:39:29.598742 13451 tablet_service.cc:1514] Processing DeleteTablet for tablet f9fb60636f27466eb0687d8b86469b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52512
I20260926 07:39:29.598773 13313 tablet_service.cc:1514] Processing DeleteTablet for tablet f9fb60636f27466eb0687d8b86469b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:40484
I20260926 07:39:29.609696 13621 tablet_replica.cc:330] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: stopping tablet replica
I20260926 07:39:29.609929 13622 tablet_replica.cc:330] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: stopping tablet replica
I20260926 07:39:29.609951 13621 raft_consensus.cc:2229] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:29.610144 13622 raft_consensus.cc:2229] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:29.610387 13622 raft_consensus.cc:2258] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.610582 13621 raft_consensus.cc:2258] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.611500 13623 tablet_replica.cc:330] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: stopping tablet replica
I20260926 07:39:29.611544 13621 ts_tablet_manager.cc:1905] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.611682 13622 ts_tablet_manager.cc:1905] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.611681 13623 raft_consensus.cc:2229] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:29.611886 13623 raft_consensus.cc:2258] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.612880 13623 ts_tablet_manager.cc:1905] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.618223 13621 ts_tablet_manager.cc:1918] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:29.618356 13621 log.cc:1196] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/f9fb60636f27466eb0687d8b86469b4f
I20260926 07:39:29.618826 13621 ts_tablet_manager.cc:1939] T f9fb60636f27466eb0687d8b86469b4f P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting consensus metadata
I20260926 07:39:29.618988 13622 ts_tablet_manager.cc:1918] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:29.619127 13622 log.cc:1196] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/f9fb60636f27466eb0687d8b86469b4f
I20260926 07:39:29.619903 13021 catalog_manager.cc:4841] TS a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983): tablet f9fb60636f27466eb0687d8b86469b4f (table test-table [id=cc813ed476414935a7b814e7c3286468]) successfully deleted
I20260926 07:39:29.620062 13622 ts_tablet_manager.cc:1939] T f9fb60636f27466eb0687d8b86469b4f P dc5873b6c8564cd5b2fe939718f81bb8: Deleting consensus metadata
I20260926 07:39:29.620263 13623 ts_tablet_manager.cc:1918] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:29.620401 13623 log.cc:1196] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/f9fb60636f27466eb0687d8b86469b4f
I20260926 07:39:29.620855 13623 ts_tablet_manager.cc:1939] T f9fb60636f27466eb0687d8b86469b4f P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting consensus metadata
I20260926 07:39:29.621062 13021 catalog_manager.cc:4841] TS dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837): tablet f9fb60636f27466eb0687d8b86469b4f (table test-table [id=cc813ed476414935a7b814e7c3286468]) successfully deleted
I20260926 07:39:29.621940 13022 catalog_manager.cc:4841] TS 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065): tablet f9fb60636f27466eb0687d8b86469b4f (table test-table [id=cc813ed476414935a7b814e7c3286468]) successfully deleted
Sep 26 07:39:29 dist-test-slave-x492 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 07:39:29 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:29.654115 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:29.639766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48162 (local address 127.11.9.126:42791)
0926 07:39:29.640546 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:29.640550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:29.640566 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:39:29.640610 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:29.640613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:29.640668 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:29.640797 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:29.642118 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:29.643003 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:29.644339 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:29.644719 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:29.644853 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:29.648831 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:29.648835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:29.648838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:29.648870 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:29.651874 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:29.652533 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:29.652536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:29.652539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:29.652642 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:29.653014 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:29.653017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:29.653018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:29.653316 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:29.653487 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:29.653718 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 07:39:29.653864 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":496,"threads_started":1}
I20260926 07:39:29.657603 13035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:29.658187 13035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:29.664587 13174 tablet_service.cc:1467] Processing CreateTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 (DEFAULT_TABLE table=test-table [id=3aa0210f98bb4235964c7fc8df31f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.664592 13313 tablet_service.cc:1467] Processing CreateTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 (DEFAULT_TABLE table=test-table [id=3aa0210f98bb4235964c7fc8df31f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.664783 13451 tablet_service.cc:1467] Processing CreateTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 (DEFAULT_TABLE table=test-table [id=3aa0210f98bb4235964c7fc8df31f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.664994 13313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de11af032044ea0b5a38f5a952ccac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.665014 13174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de11af032044ea0b5a38f5a952ccac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.665222 13451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5de11af032044ea0b5a38f5a952ccac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.669508 13632 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Bootstrap starting.
I20260926 07:39:29.671051 13632 tablet_bootstrap.cc:654] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.672364 13633 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Bootstrap starting.
I20260926 07:39:29.673396 13632 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: No bootstrap required, opened a new log
I20260926 07:39:29.673616 13632 ts_tablet_manager.cc:1397] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260926 07:39:29.674036 13633 tablet_bootstrap.cc:654] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.674487 13632 raft_consensus.cc:348] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.675789 13632 raft_consensus.cc:374] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.675890 13632 raft_consensus.cc:729] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.676126 13633 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: No bootstrap required, opened a new log
I20260926 07:39:29.676259 13633 ts_tablet_manager.cc:1397] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260926 07:39:29.676157 13632 consensus_queue.cc:260] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.676709 13632 ts_tablet_manager.cc:1428] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:29.677021 13633 raft_consensus.cc:348] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.677235 13633 raft_consensus.cc:374] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.677315 13633 raft_consensus.cc:729] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.677526 13633 consensus_queue.cc:260] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.678541 13633 ts_tablet_manager.cc:1428] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 07:39:29.675482 13634 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Bootstrap starting.
I20260926 07:39:29.680078 13634 tablet_bootstrap.cc:654] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.682245 13634 tablet_bootstrap.cc:492] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: No bootstrap required, opened a new log
I20260926 07:39:29.682382 13634 ts_tablet_manager.cc:1397] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260926 07:39:29.683153 13634 raft_consensus.cc:348] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.683342 13634 raft_consensus.cc:374] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.683408 13634 raft_consensus.cc:729] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eacecc48b6d4baca02bd2b8d76586e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.683575 13634 consensus_queue.cc:260] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.684224 13634 ts_tablet_manager.cc:1428] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:29.828660 13610 raft_consensus.cc:482] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:29.828944 13610 raft_consensus.cc:504] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.829816 13610 leader_election.cc:288] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:29.830377 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de11af032044ea0b5a38f5a952ccac6" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" is_pre_election: true
I20260926 07:39:29.830427 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de11af032044ea0b5a38f5a952ccac6" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" is_pre_election: true
I20260926 07:39:29.830651 13338 raft_consensus.cc:2454] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:29.830691 13471 raft_consensus.cc:2454] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 0.
I20260926 07:39:29.831156 13125 leader_election.cc:302] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:29.831460 13610 raft_consensus.cc:2781] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:29.831583 13610 raft_consensus.cc:482] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:29.831694 13610 raft_consensus.cc:3037] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:29.833791 13610 raft_consensus.cc:504] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.834309 13610 leader_election.cc:288] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Requested vote from peers 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:29.834774 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de11af032044ea0b5a38f5a952ccac6" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
I20260926 07:39:29.834755 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5de11af032044ea0b5a38f5a952ccac6" candidate_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
I20260926 07:39:29.834954 13471 raft_consensus.cc:3037] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:29.834975 13338 raft_consensus.cc:3037] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:29.837874 13338 raft_consensus.cc:2454] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:29.837888 13471 raft_consensus.cc:2454] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ca52f612ac43b88ce1f953b2b6cffc in term 1.
I20260926 07:39:29.838294 13125 leader_election.cc:302] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:29.838618 13610 raft_consensus.cc:2781] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:29.838763 13610 raft_consensus.cc:686] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 LEADER]: Becoming Leader. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Running, Role: LEADER
I20260926 07:39:29.839056 13610 consensus_queue.cc:237] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.841627 13036 catalog_manager.cc:5495] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65). New cstate: current_term: 1 leader_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:29.875552 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:29.870932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.11.9.65:44983)
0926 07:39:29.871462 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:29.871467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:29.871491 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:39:29.871547 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:29.871551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:29.871606 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:29.871681 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:29.871993 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:29.872733 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:29.874012 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:29.874425 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:29.874537 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:29.874880 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:29.874965 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:29.875038 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:29.875174 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 07:39:29.875226 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":294,"threads_started":1}
I20260926 07:39:29.883337 13338 raft_consensus.cc:1261] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:29.883615 13471 raft_consensus.cc:1261] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Refusing update from remote peer a6ca52f612ac43b88ce1f953b2b6cffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:29.883831 13644 consensus_queue.cc:1035] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:29.884158 13610 consensus_queue.cc:1035] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:29.930724 13037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:29.930920 13037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:29.933513 13037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Sending DeleteTablet for 3 replicas of tablet 5de11af032044ea0b5a38f5a952ccac6
I20260926 07:39:29.934365 13174 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:46086
I20260926 07:39:29.934396 13451 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52512
I20260926 07:39:29.934401 13313 tablet_service.cc:1514] Processing DeleteTablet for tablet 5de11af032044ea0b5a38f5a952ccac6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:29 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:40484
I20260926 07:39:29.934643 13622 tablet_replica.cc:330] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: stopping tablet replica
I20260926 07:39:29.934665 13623 tablet_replica.cc:330] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: stopping tablet replica
I20260926 07:39:29.934787 13622 raft_consensus.cc:2229] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:29.934854 13623 raft_consensus.cc:2229] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:29.934939 13621 tablet_replica.cc:330] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: stopping tablet replica
I20260926 07:39:29.934993 13622 raft_consensus.cc:2258] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.935030 13623 raft_consensus.cc:2258] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.935129 13621 raft_consensus.cc:2229] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:29.935392 13621 raft_consensus.cc:2258] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:29.935443 13037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:29.936167 13037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:29.936568 13622 ts_tablet_manager.cc:1905] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.937260 13623 ts_tablet_manager.cc:1905] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.941905 13621 ts_tablet_manager.cc:1905] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:29.942592 13622 ts_tablet_manager.cc:1918] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:29.942754 13622 log.cc:1196] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/5de11af032044ea0b5a38f5a952ccac6
I20260926 07:39:29.943112 13622 ts_tablet_manager.cc:1939] T 5de11af032044ea0b5a38f5a952ccac6 P dc5873b6c8564cd5b2fe939718f81bb8: Deleting consensus metadata
I20260926 07:39:29.943496 13623 ts_tablet_manager.cc:1918] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:29.943660 13623 log.cc:1196] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/5de11af032044ea0b5a38f5a952ccac6
I20260926 07:39:29.944092 13623 ts_tablet_manager.cc:1939] T 5de11af032044ea0b5a38f5a952ccac6 P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting consensus metadata
I20260926 07:39:29.944075 13021 catalog_manager.cc:4841] TS dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837): tablet 5de11af032044ea0b5a38f5a952ccac6 (table test-table [id=3aa0210f98bb4235964c7fc8df31f504]) successfully deleted
I20260926 07:39:29.945045 13022 catalog_manager.cc:4841] TS 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065): tablet 5de11af032044ea0b5a38f5a952ccac6 (table test-table [id=3aa0210f98bb4235964c7fc8df31f504]) successfully deleted
I20260926 07:39:29.945436 13451 tablet_service.cc:1467] Processing CreateTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (DEFAULT_TABLE table=test-table [id=e17690cb90f64cbda893285ad3d82361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.945494 13313 tablet_service.cc:1467] Processing CreateTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (DEFAULT_TABLE table=test-table [id=e17690cb90f64cbda893285ad3d82361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.945801 13451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.945940 13313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.950066 13634 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Bootstrap starting.
I20260926 07:39:29.950326 13174 tablet_service.cc:1467] Processing CreateTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (DEFAULT_TABLE table=test-table [id=e17690cb90f64cbda893285ad3d82361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:29.950730 13174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:29.951663 13634 tablet_bootstrap.cc:654] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.953410 13633 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Bootstrap starting.
I20260926 07:39:29.954746 13621 ts_tablet_manager.cc:1918] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:39:29.954907 13621 log.cc:1196] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/5de11af032044ea0b5a38f5a952ccac6
I20260926 07:39:29.954972 13633 tablet_bootstrap.cc:654] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.955343 13621 ts_tablet_manager.cc:1939] T 5de11af032044ea0b5a38f5a952ccac6 P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting consensus metadata
I20260926 07:39:29.956386 13021 catalog_manager.cc:4841] TS a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983): tablet 5de11af032044ea0b5a38f5a952ccac6 (table test-table [id=3aa0210f98bb4235964c7fc8df31f504]) successfully deleted
I20260926 07:39:29.959491 13633 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: No bootstrap required, opened a new log
I20260926 07:39:29.959642 13633 ts_tablet_manager.cc:1397] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 07:39:29.959807 13634 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: No bootstrap required, opened a new log
I20260926 07:39:29.959987 13634 ts_tablet_manager.cc:1397] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260926 07:39:29.960321 13633 raft_consensus.cc:348] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.960497 13632 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Bootstrap starting.
I20260926 07:39:29.960588 13633 raft_consensus.cc:374] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.960676 13633 raft_consensus.cc:729] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.960641 13634 raft_consensus.cc:348] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.960902 13634 raft_consensus.cc:374] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.960983 13634 raft_consensus.cc:729] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eacecc48b6d4baca02bd2b8d76586e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.960893 13633 consensus_queue.cc:260] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.961499 13633 ts_tablet_manager.cc:1428] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:29.961752 13634 consensus_queue.cc:260] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.962107 13632 tablet_bootstrap.cc:654] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:29.962890 13634 ts_tablet_manager.cc:1428] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:29.965061 13632 tablet_bootstrap.cc:492] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: No bootstrap required, opened a new log
I20260926 07:39:29.965288 13632 ts_tablet_manager.cc:1397] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260926 07:39:29.966027 13632 raft_consensus.cc:348] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.966233 13632 raft_consensus.cc:374] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:29.966315 13632 raft_consensus.cc:729] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ca52f612ac43b88ce1f953b2b6cffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:29.966529 13632 consensus_queue.cc:260] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:29.967008 13632 ts_tablet_manager.cc:1428] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 07:39:30.332690 13639 raft_consensus.cc:482] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:30.332965 13639 raft_consensus.cc:504] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:30.334486 13639 leader_election.cc:288] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:30.341883 13637 raft_consensus.cc:482] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:30.342135 13637 raft_consensus.cc:504] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:30.342720 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:30.335061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:45765 (local address 127.11.9.65:44983)
0926 07:39:30.335303 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:30.335310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:30.335574 (+ 264us) server_negotiation.cc:407] Connection header received
0926 07:39:30.335854 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:30.335863 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:30.335954 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:30.336055 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:30.336832 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.337953 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.340516 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.341718 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.341842 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:30.341954 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:30.342335 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 07:39:30.342459 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 07:39:30.342885 13637 leader_election.cc:288] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983), 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065)
I20260926 07:39:30.343068 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:30.335657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35837 (local address 127.11.9.66:53797)
0926 07:39:30.336283 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:30.336299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:30.336410 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:30.336738 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:30.336744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:30.336772 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:30.337278 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:30.337289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.338517 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.338522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:30.340551 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:30.340563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.342269 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.342275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:30.342453 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:30.342489 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:39:30.342602 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 07:39:30.342724 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":283,"threads_started":1}
I20260926 07:39:30.343436 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:30.334888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44983 (local address 127.11.9.66:45765)
0926 07:39:30.335458 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:30.335475 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:30.335643 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:30.336247 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:30.336251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:30.336281 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:30.336646 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:30.336657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.338089 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.338092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:30.340371 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:30.340385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.341938 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.341944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:30.342118 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:30.342160 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 07:39:30.342958 (+ 798us) client_negotiation.cc:240] Negotiation successful
0926 07:39:30.343091 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":330,"threads_started":1}
I20260926 07:39:30.343561 13197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" is_pre_election: true
I20260926 07:39:30.343724 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" is_pre_election: true
I20260926 07:39:30.343833 13197 raft_consensus.cc:2454] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 0.
I20260926 07:39:30.343977 13338 raft_consensus.cc:2454] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 0.
I20260926 07:39:30.344071 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:30.335838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:53797 (local address 127.11.9.67:35837)
0926 07:39:30.336493 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:30.336497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:30.336516 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:39:30.336570 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:30.336573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:30.336622 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:30.336701 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:30.337392 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.338389 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.340713 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.341986 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.342141 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:30.342288 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:30.342632 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 07:39:30.342756 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":423,"threads_started":1}
I20260926 07:39:30.344061 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" is_pre_election: true
I20260926 07:39:30.344271 13199 raft_consensus.cc:2454] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eacecc48b6d4baca02bd2b8d76586e5 in term 0.
I20260926 07:39:30.344321 13403 leader_election.cc:302] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ca52f612ac43b88ce1f953b2b6cffc, dc5873b6c8564cd5b2fe939718f81bb8; no voters:
I20260926 07:39:30.344578 13637 raft_consensus.cc:2781] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:30.344722 13637 raft_consensus.cc:482] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:30.344837 13637 raft_consensus.cc:3037] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:30.344844 13264 leader_election.cc:302] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5, a6ca52f612ac43b88ce1f953b2b6cffc; no voters:
I20260926 07:39:30.345010 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" is_pre_election: true
I20260926 07:39:30.345260 13639 raft_consensus.cc:2781] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:30.345379 13639 raft_consensus.cc:482] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:30.345458 13639 raft_consensus.cc:3037] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:30.347879 13637 raft_consensus.cc:504] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:30.348258 13471 raft_consensus.cc:2379] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6eacecc48b6d4baca02bd2b8d76586e5 in current term 1: Already voted for candidate dc5873b6c8564cd5b2fe939718f81bb8 in this term.
I20260926 07:39:30.348330 13639 raft_consensus.cc:504] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:30.348508 13637 leader_election.cc:288] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 election: Requested vote from peers a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983), 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065)
I20260926 07:39:30.348896 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
I20260926 07:39:30.349097 13199 raft_consensus.cc:3037] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:30.349341 13338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eacecc48b6d4baca02bd2b8d76586e5"
I20260926 07:39:30.349457 13197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc"
I20260926 07:39:30.349663 13338 raft_consensus.cc:2379] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dc5873b6c8564cd5b2fe939718f81bb8 in current term 1: Already voted for candidate 6eacecc48b6d4baca02bd2b8d76586e5 in this term.
I20260926 07:39:30.349737 13639 leader_election.cc:288] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [CANDIDATE]: Term 1 election: Requested vote from peers a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983), dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837)
I20260926 07:39:30.350340 13471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f317cbd5c6429bac7d9f7dd0f82ceb" candidate_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5873b6c8564cd5b2fe939718f81bb8"
I20260926 07:39:30.350565 13471 raft_consensus.cc:2379] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6eacecc48b6d4baca02bd2b8d76586e5 in current term 1: Already voted for candidate dc5873b6c8564cd5b2fe939718f81bb8 in this term.
I20260926 07:39:30.350878 13264 leader_election.cc:302] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eacecc48b6d4baca02bd2b8d76586e5; no voters: a6ca52f612ac43b88ce1f953b2b6cffc, dc5873b6c8564cd5b2fe939718f81bb8
I20260926 07:39:30.351159 13639 raft_consensus.cc:2726] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 07:39:30.351822 13199 raft_consensus.cc:2454] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc5873b6c8564cd5b2fe939718f81bb8 in term 1.
I20260926 07:39:30.352187 13403 leader_election.cc:302] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a6ca52f612ac43b88ce1f953b2b6cffc, dc5873b6c8564cd5b2fe939718f81bb8; no voters: 6eacecc48b6d4baca02bd2b8d76586e5
I20260926 07:39:30.352460 13637 raft_consensus.cc:2781] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:30.352828 13637 raft_consensus.cc:686] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 LEADER]: Becoming Leader. State: Replica: dc5873b6c8564cd5b2fe939718f81bb8, State: Running, Role: LEADER
I20260926 07:39:30.353127 13637 consensus_queue.cc:237] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } }
I20260926 07:39:30.355728 13036 catalog_manager.cc:5495] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc5873b6c8564cd5b2fe939718f81bb8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35837 } health_report { overall_health: HEALTHY } } }
I20260926 07:39:30.424053 13653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:30.419021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.11.9.67:35837)
0926 07:39:30.419241 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:30.419248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:30.419542 (+ 294us) server_negotiation.cc:407] Connection header received
0926 07:39:30.419733 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:30.419738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:30.419810 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:30.419903 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:30.420292 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.420984 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.422440 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:30.422853 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:30.422995 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:30.423352 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:30.423463 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:30.423560 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:30.423699 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 07:39:30.423770 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:39:30.432581 13338 raft_consensus.cc:1261] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Refusing update from remote peer dc5873b6c8564cd5b2fe939718f81bb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:30.433118 13199 raft_consensus.cc:1261] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Refusing update from remote peer dc5873b6c8564cd5b2fe939718f81bb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:30.433063 13654 consensus_queue.cc:1035] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eacecc48b6d4baca02bd2b8d76586e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:30.433653 13654 consensus_queue.cc:1035] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ca52f612ac43b88ce1f953b2b6cffc" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:30.438498 13606 mvcc.cc:205] Tried to move back new op lower bound from 7333512685287784448 to 7333512684968681472. Current Snapshot: MvccSnapshot[applied={T|T < 7333512685287784448}]
W20260926 07:39:30.451659 13605 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 07:39:32.029287 13036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:32.029505 13036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:32.033085 13036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be62022654b643548957295598c54e7e: Sending DeleteTablet for 3 replicas of tablet 86f317cbd5c6429bac7d9f7dd0f82ceb
I20260926 07:39:32.033784 13174 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:32 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:46086
I20260926 07:39:32.033980 13451 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:32 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52512
I20260926 07:39:32.033980 13313 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f317cbd5c6429bac7d9f7dd0f82ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:32 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:40484
I20260926 07:39:32.034371 13686 tablet_replica.cc:330] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: stopping tablet replica
I20260926 07:39:32.034613 13686 raft_consensus.cc:2229] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:32.034889 13686 raft_consensus.cc:2258] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:32.034960 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13108
I20260926 07:39:32.036073 13686 ts_tablet_manager.cc:1905] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:32.044301 13686 ts_tablet_manager.cc:1918] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:32.044481 13686 log.cc:1196] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/86f317cbd5c6429bac7d9f7dd0f82ceb
I20260926 07:39:32.045043 13686 ts_tablet_manager.cc:1939] T 86f317cbd5c6429bac7d9f7dd0f82ceb P a6ca52f612ac43b88ce1f953b2b6cffc: Deleting consensus metadata
I20260926 07:39:32.046303 13021 catalog_manager.cc:4841] TS a6ca52f612ac43b88ce1f953b2b6cffc (127.11.9.65:44983): tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (table test-table [id=e17690cb90f64cbda893285ad3d82361]) successfully deleted
I20260926 07:39:32.046670 13687 tablet_replica.cc:330] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: stopping tablet replica
I20260926 07:39:32.046941 13687 raft_consensus.cc:2229] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:32.047370 13687 raft_consensus.cc:2258] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:32.048033 13688 tablet_replica.cc:330] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: stopping tablet replica
I20260926 07:39:32.048243 13688 raft_consensus.cc:2229] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:32.048831 13687 ts_tablet_manager.cc:1905] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:32.049371 13688 raft_consensus.cc:2258] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:32.050760 13688 ts_tablet_manager.cc:1905] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:32.055327 13687 ts_tablet_manager.cc:1918] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:32.055514 13687 log.cc:1196] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/86f317cbd5c6429bac7d9f7dd0f82ceb
I20260926 07:39:32.055925 13687 ts_tablet_manager.cc:1939] T 86f317cbd5c6429bac7d9f7dd0f82ceb P dc5873b6c8564cd5b2fe939718f81bb8: Deleting consensus metadata
I20260926 07:39:32.056998 13021 catalog_manager.cc:4841] TS dc5873b6c8564cd5b2fe939718f81bb8 (127.11.9.67:35837): tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (table test-table [id=e17690cb90f64cbda893285ad3d82361]) successfully deleted
I20260926 07:39:32.057150 13688 ts_tablet_manager.cc:1918] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 07:39:32.057325 13688 log.cc:1196] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/86f317cbd5c6429bac7d9f7dd0f82ceb
I20260926 07:39:32.057714 13688 ts_tablet_manager.cc:1939] T 86f317cbd5c6429bac7d9f7dd0f82ceb P 6eacecc48b6d4baca02bd2b8d76586e5: Deleting consensus metadata
I20260926 07:39:32.058580 13022 catalog_manager.cc:4841] TS 6eacecc48b6d4baca02bd2b8d76586e5 (127.11.9.66:46065): tablet 86f317cbd5c6429bac7d9f7dd0f82ceb (table test-table [id=e17690cb90f64cbda893285ad3d82361]) successfully deleted
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:32.065119 13690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:32.036521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.11.9.65:44983)
0926 07:39:32.050383 (+ 13862us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:32.050389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:32.050706 (+ 317us) server_negotiation.cc:407] Connection header received
0926 07:39:32.050755 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:32.050758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:32.050812 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:32.050933 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:32.052372 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.053204 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.054417 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.054848 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.055022 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:32.059644 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:32.059658 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:32.059675 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:32.059709 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:32.062699 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.063358 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.063368 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.063371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.063505 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.063834 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.063837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.063839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.064172 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:32.064389 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:32.064625 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 07:39:32.064776 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13728,"thread_start_us":13613,"threads_started":1}
W20260926 07:39:32.065783 13239 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120
W20260926 07:39:32.066334 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13108: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:32.066545 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13108
I20260926 07:39:32.092219 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13248
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:32.106345 13691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:32.092910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.11.9.66:46065)
0926 07:39:32.093415 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:32.093419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:32.093433 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:39:32.093482 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:32.093485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:32.093538 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:32.093650 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:32.094831 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.095541 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.096790 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.097260 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.097390 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:32.101426 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:32.101436 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:32.101452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:32.101487 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:32.104026 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.104640 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.104651 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.104657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.104786 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.105087 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.105090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.105092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.105489 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:32.105689 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:32.105926 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 07:39:32.106088 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":270,"threads_started":1}
W20260926 07:39:32.107048 13378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40486
W20260926 07:39:32.107635 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13248: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:32.107779 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13248
I20260926 07:39:32.125156 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13387
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:32.144524 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:32.125944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.11.9.67:35837)
0926 07:39:32.126524 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:32.126530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:32.126550 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:39:32.126613 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:32.126618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:32.126687 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:32.126789 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:32.128064 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.129229 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.130390 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:32.131196 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:32.131317 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:32.137140 (+ 5823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:32.137179 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:32.137204 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:32.137274 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:32.141540 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.142317 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.142328 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.142334 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.142518 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:32.142948 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:32.142953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:32.142958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:32.143456 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:32.143790 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:32.144059 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 07:39:32.144268 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":285,"threads_started":1}
W20260926 07:39:32.145506 13517 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52536
W20260926 07:39:32.146276 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13387: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:32.146463 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13387
I20260926 07:39:32.168016 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13005
I20260926 07:39:32.169061 13077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:32.283072 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13005
2026-09-26T07:39:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8465 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[13696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[13696](info): set up 2 sockets
Sep 26 07:39:32 dist-test-slave-x492 krb5kdc[13696](info): commencing operation
krb5kdc: starting...
W20260926 07:39:33.385598 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.006s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:33 dist-test-slave-x492 krb5kdc[13696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:33.562783 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45487
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:33.873286 13709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:33.873538 13709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:33.873706 13709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:33.880604 13709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:33.880693 13709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:33.880792 13709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:33.880882 13709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:33.880960 13709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:33.890486 13709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:33.891703 13709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:33.892967 13709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:33.904364 13717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:33.904482 13714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:33.904464 13715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:33.905035 13709 server_base.cc:1034] running on GCE node
I20260926 07:39:33.905812 13709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:33.907071 13709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:33.908283 13709 hybrid_clock.cc:648] HybridClock initialized: now 1790408373908243 us; error 35 us; skew 500 ppm
I20260926 07:39:33.908610 13709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:33 dist-test-slave-x492 krb5kdc[13696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408373, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:33.911558 13709 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:33.912950 13709 webserver.cc:460] Webserver started at http://127.11.9.126:46839/ using document root <none> and password file <none>
I20260926 07:39:33.913659 13709 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:33.913796 13709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:33.914139 13709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:33.916978 13709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "049c6e2b94464e27b74431b42f247a81"
format_stamp: "Formatted at 2026-09-26 07:39:33 on dist-test-slave-x492"
server_key: "97fe3a7e3eb2ff4e341a22101a87e6e8"
server_key_iv: "d51d7080bcca628ad53296752292a8ec"
server_key_version: "encryptionkey@0"
I20260926 07:39:33.917829 13709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "049c6e2b94464e27b74431b42f247a81"
format_stamp: "Formatted at 2026-09-26 07:39:33 on dist-test-slave-x492"
server_key: "97fe3a7e3eb2ff4e341a22101a87e6e8"
server_key_iv: "d51d7080bcca628ad53296752292a8ec"
server_key_version: "encryptionkey@0"
I20260926 07:39:33.922809 13709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:33.926043 13724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:33.927325 13709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:33.927495 13709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "049c6e2b94464e27b74431b42f247a81"
format_stamp: "Formatted at 2026-09-26 07:39:33 on dist-test-slave-x492"
server_key: "97fe3a7e3eb2ff4e341a22101a87e6e8"
server_key_iv: "d51d7080bcca628ad53296752292a8ec"
server_key_version: "encryptionkey@0"
I20260926 07:39:33.927690 13709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:33.949568 13709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:33.953924 13709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:33.954200 13709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:33.972129 13709 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45487
I20260926 07:39:33.972148 13775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45487 every 8 connection(s)
I20260926 07:39:33.973639 13709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:33.978487 13776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:33.978866 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13709
I20260926 07:39:33.979091 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:33.979946 11301 external_mini_cluster.cc:1428] Setting key bdd410541498d5641e30083a30adccc2
I20260926 07:39:33.992214 13776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81: Bootstrap starting.
Sep 26 07:39:33 dist-test-slave-x492 krb5kdc[13696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:33.995141 13776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:33.996294 13776 log.cc:824] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:33.998967 13776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81: No bootstrap required, opened a new log
I20260926 07:39:34.001713 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:33.981919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.11.9.126:45487)
0926 07:39:33.982779 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:33.982791 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:33.982838 (+ 47us) server_negotiation.cc:407] Connection header received
0926 07:39:33.985209 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:33.985242 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:33.985656 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:33.986048 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:33.987489 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:33.988774 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:33.990111 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:33.990797 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:33.990983 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:33.994746 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:33.994753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:33.994767 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:33.994806 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:33.997895 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:33.998518 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:33.998524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:33.998530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:33.998686 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:33.999048 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:33.999052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:33.999066 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:33.999566 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:33.999883 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:34.000294 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 07:39:34.000699 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":366,"threads_started":1}
I20260926 07:39:34.008162 13776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } }
I20260926 07:39:34.008517 13776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:34.008615 13776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049c6e2b94464e27b74431b42f247a81, State: Initialized, Role: FOLLOWER
I20260926 07:39:34.009131 13776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } }
I20260926 07:39:34.009416 13776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:34.009555 13776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:34.009724 13776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:34.011807 13776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } }
I20260926 07:39:34.012295 13776 leader_election.cc:302] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049c6e2b94464e27b74431b42f247a81; no voters:
I20260926 07:39:34.013041 13776 leader_election.cc:288] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:34.013147 13781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:34.013741 13781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [term 1 LEADER]: Becoming Leader. State: Replica: 049c6e2b94464e27b74431b42f247a81, State: Running, Role: LEADER
I20260926 07:39:34.014362 13781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } }
I20260926 07:39:34.015252 13776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:34.017403 13783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049c6e2b94464e27b74431b42f247a81. Latest consensus state: current_term: 1 leader_uuid: "049c6e2b94464e27b74431b42f247a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } } }
I20260926 07:39:34.017390 13782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "049c6e2b94464e27b74431b42f247a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049c6e2b94464e27b74431b42f247a81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45487 } } }
I20260926 07:39:34.017849 13783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:34.017872 13782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:34.019850 13786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:34.025022 13786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:34.034449 13786 catalog_manager.cc:1347] Generated new cluster ID: 5599257a3a0f47e3aa5faa3a416bf9d0
I20260926 07:39:34.034559 13786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:34.062716 13786 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:34.063971 13786 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:34.072201 13786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 049c6e2b94464e27b74431b42f247a81: Generated new TSK 0
I20260926 07:39:34.072957 13786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:34.128471 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:34.443616 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:34.443881 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:34.444041 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:34.451155 13802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:34.451274 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:34.451448 13802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:34.460800 13802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:34.462028 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:34.463587 13802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:34.477749 13807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:34.477789 13810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:34.477818 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:34.478525 13802 server_base.cc:1034] running on GCE node
I20260926 07:39:34.479045 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:34.479892 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:34.481096 13802 hybrid_clock.cc:648] HybridClock initialized: now 1790408374481050 us; error 54 us; skew 500 ppm
I20260926 07:39:34.481441 13802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:34 dist-test-slave-x492 krb5kdc[13696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408374, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:34.484488 13802 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:34.485852 13802 webserver.cc:460] Webserver started at http://127.11.9.65:35795/ using document root <none> and password file <none>
I20260926 07:39:34.486475 13802 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:34.486570 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:34.486814 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:34.489542 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "1e60f17d86b84a3f934d39ec9634cf86"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "2901f425225e971bce2685a5bde0663d"
server_key_iv: "88022b35791dcc25821713d9868055f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.490262 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "1e60f17d86b84a3f934d39ec9634cf86"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "2901f425225e971bce2685a5bde0663d"
server_key_iv: "88022b35791dcc25821713d9868055f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.495023 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:39:34.498188 13817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:34.499408 13802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:39:34.499532 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "1e60f17d86b84a3f934d39ec9634cf86"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "2901f425225e971bce2685a5bde0663d"
server_key_iv: "88022b35791dcc25821713d9868055f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.499680 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:34.517092 13802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:34.520934 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:34.521235 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:34.522111 13802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:34.523401 13802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:34.523496 13802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:34.523638 13802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:34.523708 13802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:34.555675 13802 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:35031
I20260926 07:39:34.555751 13929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:35031 every 8 connection(s)
I20260926 07:39:34.556883 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:34.565780 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13802
I20260926 07:39:34.565984 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:34.566560 11301 external_mini_cluster.cc:1428] Setting key 032bde0f0874bd31e40caf8f97ca4c17
Sep 26 07:39:34 dist-test-slave-x492 krb5kdc[13696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408374, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:34.576812 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:34.559315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:52431 (local address 127.11.9.126:45487)
0926 07:39:34.559755 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:34.559759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:34.560539 (+ 780us) server_negotiation.cc:407] Connection header received
0926 07:39:34.562410 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:34.562415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:34.562509 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:34.562632 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:34.564102 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:34.564828 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:34.566537 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:34.566996 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:34.567134 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:34.570893 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:34.570902 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:34.570910 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:34.570952 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:34.573928 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:34.574658 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:34.574672 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:34.574674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:34.574783 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:34.575188 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:34.575193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:34.575196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:34.575457 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:34.575607 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:34.576236 (+ 629us) server_negotiation.cc:299] Negotiation successful
0926 07:39:34.576446 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":250,"threads_started":1}
I20260926 07:39:34.577790 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:34.559739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45487 (local address 127.11.9.65:52431)
0926 07:39:34.560387 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:34.560423 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:34.562150 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:34.562879 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:34.562888 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:34.563392 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:34.563939 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:34.563955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:34.564990 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:34.564994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:34.566401 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:34.566412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:34.567233 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:34.567238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:34.567532 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:34.568178 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:34.568203 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:34.570675 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:34.574065 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:34.574074 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:34.574090 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:34.574530 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:34.574886 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:34.574892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:34.574896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:34.575074 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:34.575583 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:34.575590 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:34.575904 (+ 314us) client_negotiation.cc:769] Sending connection context
0926 07:39:34.576152 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 07:39:34.576661 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":272,"threads_started":1}
I20260926 07:39:34.579288 13930 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45487
I20260926 07:39:34.579628 13930 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:34.580523 13930 heartbeater.cc:511] Master 127.11.9.126:45487 requested a full tablet report, sending...
I20260926 07:39:34.582988 13741 ts_manager.cc:194] Registered new tserver with Master: 1e60f17d86b84a3f934d39ec9634cf86 (127.11.9.65:35031)
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:39:34.584872 13741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:52431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:34.606330 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:34.918937 13936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:34.919247 13936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:34.919409 13936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:34.926705 13936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:34.926858 13936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:34.927023 13936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:34.937301 13936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:34.938620 13936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:34.939967 13936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:34.953518 13941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:34.953437 13944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:34.953438 13942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:34.954651 13936 server_base.cc:1034] running on GCE node
I20260926 07:39:34.955201 13936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:34.956060 13936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:34.957264 13936 hybrid_clock.cc:648] HybridClock initialized: now 1790408374957203 us; error 47 us; skew 500 ppm
I20260926 07:39:34.957569 13936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:34 dist-test-slave-x492 krb5kdc[13696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408374, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:34.960701 13936 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:34.962116 13936 webserver.cc:460] Webserver started at http://127.11.9.66:36313/ using document root <none> and password file <none>
I20260926 07:39:34.962783 13936 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:34.962921 13936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:34.963243 13936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:34.966090 13936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "9864d78e122d43d38e1b8793c8dcd936"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "9f8fec4fec14c9f166b4d06a628971fc"
server_key_iv: "f48299db8ef019e4ccdcd03a9c0746e8"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.966854 13936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "9864d78e122d43d38e1b8793c8dcd936"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "9f8fec4fec14c9f166b4d06a628971fc"
server_key_iv: "f48299db8ef019e4ccdcd03a9c0746e8"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.971815 13936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:34.975332 13951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:34.976645 13936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:34.976805 13936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "9864d78e122d43d38e1b8793c8dcd936"
format_stamp: "Formatted at 2026-09-26 07:39:34 on dist-test-slave-x492"
server_key: "9f8fec4fec14c9f166b4d06a628971fc"
server_key_iv: "f48299db8ef019e4ccdcd03a9c0746e8"
server_key_version: "encryptionkey@0"
I20260926 07:39:34.977017 13936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:35.010536 13936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:35.015156 13936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:35.015517 13936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:35.016494 13936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:35.017863 13936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:35.017932 13936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:35.018019 13936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:35.018064 13936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:35.049666 13936 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:35443
I20260926 07:39:35.049721 14063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:35443 every 8 connection(s)
I20260926 07:39:35.050966 13936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:35.052289 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 13936
I20260926 07:39:35.052495 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:35.053067 11301 external_mini_cluster.cc:1428] Setting key b5a5c665c63ee3db4c9efa4048a35bd6
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[13696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408374, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:35.072024 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:35.053769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:50355 (local address 127.11.9.126:45487)
0926 07:39:35.053985 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:35.053992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:35.055397 (+ 1405us) server_negotiation.cc:407] Connection header received
0926 07:39:35.057225 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:35.057230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:35.057311 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:35.057422 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:35.058942 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.060102 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.061778 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.062543 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.062682 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:35.066508 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:35.066513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:35.066516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:35.066546 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:35.069118 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:35.070018 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:35.070022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:35.070024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:35.070181 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:35.070608 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:35.070611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:35.070613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:35.070842 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:35.070945 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:35.071633 (+ 688us) server_negotiation.cc:299] Negotiation successful
0926 07:39:35.071795 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
I20260926 07:39:35.073447 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:35.054431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45487 (local address 127.11.9.66:50355)
0926 07:39:35.055256 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:35.055294 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:35.056947 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:35.057649 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:35.057657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:35.058145 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:35.058784 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:35.058801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.060261 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.060265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:35.061639 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:35.061651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.062754 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.062759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:35.062938 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:35.063583 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:35.063616 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:35.066298 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:35.069302 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:35.069314 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:35.069333 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:35.069910 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:35.070291 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:35.070297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:35.070304 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:35.070512 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:35.070941 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:35.070947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:35.071324 (+ 377us) client_negotiation.cc:769] Sending connection context
0926 07:39:35.071585 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 07:39:35.072123 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":365,"threads_started":1}
I20260926 07:39:35.074862 14064 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45487
I20260926 07:39:35.075206 14064 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:35.076041 14064 heartbeater.cc:511] Master 127.11.9.126:45487 requested a full tablet report, sending...
I20260926 07:39:35.077684 13741 ts_manager.cc:194] Registered new tserver with Master: 9864d78e122d43d38e1b8793c8dcd936 (127.11.9.66:35443)
I20260926 07:39:35.078931 13741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:50355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:35.094476 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:35.408121 14069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:35.408380 14069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:35.408468 14069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:35.415508 14069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:35.415611 14069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:35.415729 14069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:35.425294 14069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:35.426462 14069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:35.427706 14069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:35.440361 14074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:35.440424 14075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:35.440748 14077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:35.441421 14069 server_base.cc:1034] running on GCE node
I20260926 07:39:35.442014 14069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:35.442888 14069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:35.444103 14069 hybrid_clock.cc:648] HybridClock initialized: now 1790408375444070 us; error 60 us; skew 500 ppm
I20260926 07:39:35.444410 14069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[13696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408375, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:35.447631 14069 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:35.449010 14069 webserver.cc:460] Webserver started at http://127.11.9.67:33213/ using document root <none> and password file <none>
I20260926 07:39:35.449728 14069 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:35.449862 14069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:35.450152 14069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:35.453076 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "6b1044728b92454daba1dbda5f77b440"
format_stamp: "Formatted at 2026-09-26 07:39:35 on dist-test-slave-x492"
server_key: "4dbc233195f1e353a2b72b73ca03cf04"
server_key_iv: "8d11267763eb2616113646a89677eb63"
server_key_version: "encryptionkey@0"
I20260926 07:39:35.453905 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "6b1044728b92454daba1dbda5f77b440"
format_stamp: "Formatted at 2026-09-26 07:39:35 on dist-test-slave-x492"
server_key: "4dbc233195f1e353a2b72b73ca03cf04"
server_key_iv: "8d11267763eb2616113646a89677eb63"
server_key_version: "encryptionkey@0"
I20260926 07:39:35.458968 14069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 07:39:35.462337 14084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:35.463560 14069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:35.463733 14069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "6b1044728b92454daba1dbda5f77b440"
format_stamp: "Formatted at 2026-09-26 07:39:35 on dist-test-slave-x492"
server_key: "4dbc233195f1e353a2b72b73ca03cf04"
server_key_iv: "8d11267763eb2616113646a89677eb63"
server_key_version: "encryptionkey@0"
I20260926 07:39:35.463920 14069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:35.480376 14069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:35.484158 14069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:35.484426 14069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:35.485352 14069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:35.486656 14069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:35.486865 14069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:35.487021 14069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:35.487093 14069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:35.518970 14069 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:42945
I20260926 07:39:35.519004 14196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:42945 every 8 connection(s)
I20260926 07:39:35.520190 14069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:35.520793 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14069
I20260926 07:39:35.521008 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:35.521721 11301 external_mini_cluster.cc:1428] Setting key 6796091bbfdbc979889d0159e029e52e
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[13696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408375, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:35.539729 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:35.523186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:42871 (local address 127.11.9.126:45487)
0926 07:39:35.523367 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:35.523371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:35.525025 (+ 1654us) server_negotiation.cc:407] Connection header received
0926 07:39:35.526768 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:35.526772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:35.526864 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:35.526950 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:35.528266 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.529005 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.530519 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.530995 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.531094 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:35.534357 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:35.534362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:35.534366 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:35.534396 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:35.536965 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:35.537667 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:35.537672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:35.537681 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:35.537790 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:35.538182 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:35.538199 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:35.538201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:35.538444 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:35.538590 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:35.539206 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 07:39:35.539406 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:39:35.540591 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:35.523914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45487 (local address 127.11.9.67:42871)
0926 07:39:35.524855 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:35.524929 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:35.526530 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:35.527121 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:35.527129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:35.527600 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:35.528132 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:35.528148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.529150 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.529153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:35.530392 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:35.530403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:35.531197 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:35.531204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:35.531379 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:35.532042 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:35.532063 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:35.534178 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:35.537102 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:35.537111 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:35.537124 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:35.537546 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:35.537894 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:35.537900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:35.537904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:35.538070 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:35.538562 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:35.538570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:35.538874 (+ 304us) client_negotiation.cc:769] Sending connection context
0926 07:39:35.539117 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 07:39:35.539567 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":338,"threads_started":1}
I20260926 07:39:35.541754 14197 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45487
I20260926 07:39:35.542025 14197 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:35.542722 14197 heartbeater.cc:511] Master 127.11.9.126:45487 requested a full tablet report, sending...
I20260926 07:39:35.544018 13741 ts_manager.cc:194] Registered new tserver with Master: 6b1044728b92454daba1dbda5f77b440 (127.11.9.67:42945)
I20260926 07:39:35.544787 13741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:42871
I20260926 07:39:35.548552 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:35.587958 13930 heartbeater.cc:503] Master 127.11.9.126:45487 was elected leader, sending a full tablet report...
W20260926 07:39:35.590466 13932 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:35.587418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.11.9.126:45487)
0926 07:39:35.587628 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:35.587638 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:35.588184 (+ 546us) server_negotiation.cc:407] Connection header received
0926 07:39:35.588400 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:35.588405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:35.588492 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:35.588598 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:35.590207 (+ 1609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38674: BlockingRecv error: recv got EOF from 127.0.0.1:38674 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:39:35.594444 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13802
W20260926 07:39:35.597433 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13802: Not authorized: Client connection negotiation failed: client connection to 127.11.9.65:35031: server requires authentication, but client does not have Kerberos credentials available
I20260926 07:39:35.597602 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13802
I20260926 07:39:35.619506 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13936
W20260926 07:39:35.622395 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13936: Not authorized: Client connection negotiation failed: client connection to 127.11.9.66:35443: server requires authentication, but client does not have Kerberos credentials available
I20260926 07:39:35.622581 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13936
I20260926 07:39:35.638051 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14069
W20260926 07:39:35.640683 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14069: Not authorized: Client connection negotiation failed: client connection to 127.11.9.67:42945: server requires authentication, but client does not have Kerberos credentials available
I20260926 07:39:35.640821 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14069
I20260926 07:39:35.655762 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 13709
I20260926 07:39:35.656883 13771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:35.757225 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 13709
2026-09-26T07:39:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3469 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[14214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[14214](info): set up 2 sockets
Sep 26 07:39:35 dist-test-slave-x492 krb5kdc[14214](info): commencing operation
krb5kdc: starting...
W20260926 07:39:36.837550 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:36 dist-test-slave-x492 krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:37.020767 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42303
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:37.330976 14227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:37.331290 14227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:37.331451 14227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:37.338394 14227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:37.338536 14227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:37.338636 14227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:37.338733 14227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:37.338804 14227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:37.348091 14227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:37.349327 14227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:37.350605 14227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:37.361624 14232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:37.361596 14233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:37.361653 14235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:37.362385 14227 server_base.cc:1034] running on GCE node
I20260926 07:39:37.363094 14227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:37.364341 14227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:37.365518 14227 hybrid_clock.cc:648] HybridClock initialized: now 1790408377365469 us; error 48 us; skew 500 ppm
I20260926 07:39:37.365861 14227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:37 dist-test-slave-x492 krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408377, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:37.373626 14227 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:37.375001 14227 webserver.cc:460] Webserver started at http://127.11.9.126:41073/ using document root <none> and password file <none>
I20260926 07:39:37.375645 14227 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:37.375782 14227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:37.376084 14227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:37.378994 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "aca57f8a8776470481cf48087dd8cb53"
format_stamp: "Formatted at 2026-09-26 07:39:37 on dist-test-slave-x492"
server_key: "29f4de0813eb4426b09896a4dd423031"
server_key_iv: "fdb86c77117905e92d70753bb38f24b0"
server_key_version: "encryptionkey@0"
I20260926 07:39:37.379760 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "aca57f8a8776470481cf48087dd8cb53"
format_stamp: "Formatted at 2026-09-26 07:39:37 on dist-test-slave-x492"
server_key: "29f4de0813eb4426b09896a4dd423031"
server_key_iv: "fdb86c77117905e92d70753bb38f24b0"
server_key_version: "encryptionkey@0"
I20260926 07:39:37.384579 14227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 07:39:37.387830 14242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:37.389060 14227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 07:39:37.389272 14227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "aca57f8a8776470481cf48087dd8cb53"
format_stamp: "Formatted at 2026-09-26 07:39:37 on dist-test-slave-x492"
server_key: "29f4de0813eb4426b09896a4dd423031"
server_key_iv: "fdb86c77117905e92d70753bb38f24b0"
server_key_version: "encryptionkey@0"
I20260926 07:39:37.389456 14227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:37.410636 14227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:37.414443 14227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:37.414722 14227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:37.432720 14227 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42303
I20260926 07:39:37.432760 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42303 every 8 connection(s)
I20260926 07:39:37.434052 14227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:37.437011 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14227
I20260926 07:39:37.437288 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:37.437906 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:37.437908 11301 external_mini_cluster.cc:1428] Setting key 03def42239c16e0c9ab2bc8ef7681a1b
I20260926 07:39:37.451915 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53: Bootstrap starting.
Sep 26 07:39:37 dist-test-slave-x492 krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:37.454941 14294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:37.456009 14294 log.cc:824] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:37.458662 14294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53: No bootstrap required, opened a new log
I20260926 07:39:37.460081 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:37.439618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.11.9.126:42303)
0926 07:39:37.440443 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:37.440454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:37.440495 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:39:37.442370 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:37.442390 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:37.442779 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:37.443088 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:37.444897 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:37.446213 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:37.447918 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:37.448595 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:37.448725 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:37.453252 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:37.453261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:37.453277 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:37.453309 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:37.456357 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:37.456986 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:37.457001 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:37.457007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:37.457153 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:37.457552 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:37.457558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:37.457562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:37.458046 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:37.458314 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:37.458805 (+ 491us) server_negotiation.cc:299] Negotiation successful
0926 07:39:37.459177 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":369,"threads_started":1}
I20260926 07:39:37.463593 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:39:37.466246 14294 raft_consensus.cc:348] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } }
I20260926 07:39:37.466593 14294 raft_consensus.cc:374] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:37.466691 14294 raft_consensus.cc:729] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca57f8a8776470481cf48087dd8cb53, State: Initialized, Role: FOLLOWER
I20260926 07:39:37.467259 14294 consensus_queue.cc:260] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } }
I20260926 07:39:37.467489 14294 raft_consensus.cc:388] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:37.467619 14294 raft_consensus.cc:482] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:37.467782 14294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:37.470482 14294 raft_consensus.cc:504] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } }
I20260926 07:39:37.471024 14294 leader_election.cc:302] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aca57f8a8776470481cf48087dd8cb53; no voters:
I20260926 07:39:37.471741 14294 leader_election.cc:288] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:37.471786 14300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:37.473449 14300 raft_consensus.cc:686] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [term 1 LEADER]: Becoming Leader. State: Replica: aca57f8a8776470481cf48087dd8cb53, State: Running, Role: LEADER
I20260926 07:39:37.473958 14294 sys_catalog.cc:564] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:37.474035 14300 consensus_queue.cc:237] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } }
I20260926 07:39:37.476572 14302 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aca57f8a8776470481cf48087dd8cb53. Latest consensus state: current_term: 1 leader_uuid: "aca57f8a8776470481cf48087dd8cb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } } }
I20260926 07:39:37.476926 14302 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:37.477417 14301 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aca57f8a8776470481cf48087dd8cb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca57f8a8776470481cf48087dd8cb53" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42303 } } }
I20260926 07:39:37.477679 14301 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:37.480460 14306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:37.486605 14306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:37.509400 14306 catalog_manager.cc:1347] Generated new cluster ID: d5008e2b8fe14a8bb9179417a0cd571f
I20260926 07:39:37.509534 14306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:37.545017 14323 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 07:39:37.545552 14306 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:37.547614 14306 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:37.554287 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:37.541085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.11.9.126:42303)
0926 07:39:37.541344 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:37.541354 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:37.546056 (+ 4702us) server_negotiation.cc:407] Connection header received
0926 07:39:37.546280 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:37.546288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:37.546380 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:37.546496 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:37.547711 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:37.548897 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:37.550648 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:37.551286 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:37.551410 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:37.552534 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:37.552540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:37.552545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:37.553432 (+ 887us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:37.553483 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:37.553523 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:37.553637 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:37.553796 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 07:39:37.553917 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260926 07:39:37.556526 14306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aca57f8a8776470481cf48087dd8cb53: Generated new TSK 0
I20260926 07:39:37.557327 14306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:39:37.566732 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14227
I20260926 07:39:37.567873 14289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:37.656518 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14227
2026-09-26T07:39:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1893 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:37 dist-test-slave-x492 krb5kdc[14328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:37 dist-test-slave-x492 krb5kdc[14328](info): set up 2 sockets
Sep 26 07:39:37 dist-test-slave-x492 krb5kdc[14328](info): commencing operation
krb5kdc: starting...
W20260926 07:39:38.733619 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:38 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:38.916761 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34503
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:34503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:39.231779 14341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:39.232049 14341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:39.232219 14341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:39.239153 14341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:39.239245 14341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:39.239365 14341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:39.239457 14341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:39.239532 14341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:39.248643 14341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:34503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:39.249926 14341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:39.251183 14341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:39.262229 14346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:39.262287 14349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:39.262234 14347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:39.263135 14341 server_base.cc:1034] running on GCE node
I20260926 07:39:39.263844 14341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:39.265133 14341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:39.266371 14341 hybrid_clock.cc:648] HybridClock initialized: now 1790408379266335 us; error 26 us; skew 500 ppm
I20260926 07:39:39.266732 14341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:39 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408379, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:39.269647 14341 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:39.271049 14341 webserver.cc:460] Webserver started at http://127.11.9.126:38667/ using document root <none> and password file <none>
I20260926 07:39:39.271670 14341 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:39.271806 14341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:39.272090 14341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:39.274864 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "add86f561fb1454f814759df48c52393"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "4bb8da2cea4e7f2bf60067fc540cfcf1"
server_key_iv: "988faee4109c99733c03302c705680eb"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.275575 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "add86f561fb1454f814759df48c52393"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "4bb8da2cea4e7f2bf60067fc540cfcf1"
server_key_iv: "988faee4109c99733c03302c705680eb"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.280447 14341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260926 07:39:39.283577 14356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:39.284801 14341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:39.284967 14341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "add86f561fb1454f814759df48c52393"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "4bb8da2cea4e7f2bf60067fc540cfcf1"
server_key_iv: "988faee4109c99733c03302c705680eb"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.285141 14341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:39.326017 14341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:39.329902 14341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:39.330196 14341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:39.347390 14341 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:34503
I20260926 07:39:39.347421 14407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:34503 every 8 connection(s)
I20260926 07:39:39.348649 14341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:39.352260 14408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:39.353466 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14341
I20260926 07:39:39.353675 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:39.354485 11301 external_mini_cluster.cc:1428] Setting key 6192f006c0645501dc2a4dd67e26d6db
I20260926 07:39:39.366312 14408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393: Bootstrap starting.
Sep 26 07:39:39 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:39.369720 14408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:39.370895 14408 log.cc:824] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:39.373807 14408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393: No bootstrap required, opened a new log
I20260926 07:39:39.376142 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:39.356512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.11.9.126:34503)
0926 07:39:39.357303 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:39.357313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:39.357355 (+ 42us) server_negotiation.cc:407] Connection header received
0926 07:39:39.358841 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:39.358861 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:39.359227 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:39.359534 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:39.361143 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.362462 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.364240 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.364854 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.364978 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:39.369358 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:39.369365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:39.369379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:39.369414 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:39.372381 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:39.373026 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:39.373031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:39.373037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:39.373261 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:39.373629 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:39.373634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:39.373638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:39.374143 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:39.374475 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:39.374827 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 07:39:39.375215 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":372,"threads_started":1}
I20260926 07:39:39.381757 14408 raft_consensus.cc:348] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } }
I20260926 07:39:39.382095 14408 raft_consensus.cc:374] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:39.382210 14408 raft_consensus.cc:729] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add86f561fb1454f814759df48c52393, State: Initialized, Role: FOLLOWER
I20260926 07:39:39.382737 14408 consensus_queue.cc:260] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } }
I20260926 07:39:39.382957 14408 raft_consensus.cc:388] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:39.383095 14408 raft_consensus.cc:482] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:39.383258 14408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:39.386162 14408 raft_consensus.cc:504] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } }
I20260926 07:39:39.386654 14408 leader_election.cc:302] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: add86f561fb1454f814759df48c52393; no voters:
I20260926 07:39:39.387193 14408 leader_election.cc:288] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:39.387332 14413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:39.387976 14413 raft_consensus.cc:686] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [term 1 LEADER]: Becoming Leader. State: Replica: add86f561fb1454f814759df48c52393, State: Running, Role: LEADER
I20260926 07:39:39.388468 14413 consensus_queue.cc:237] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } }
I20260926 07:39:39.389379 14408 sys_catalog.cc:564] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:39.391340 14414 sys_catalog.cc:455] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "add86f561fb1454f814759df48c52393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } } }
I20260926 07:39:39.391571 14415 sys_catalog.cc:455] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader add86f561fb1454f814759df48c52393. Latest consensus state: current_term: 1 leader_uuid: "add86f561fb1454f814759df48c52393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add86f561fb1454f814759df48c52393" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34503 } } }
I20260926 07:39:39.391803 14414 sys_catalog.cc:458] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:39.391889 14415 sys_catalog.cc:458] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:39.393366 14418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:39.398743 14418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:39.407241 14418 catalog_manager.cc:1347] Generated new cluster ID: a3829a7792ac4cea8264c5442caaa206
I20260926 07:39:39.407349 14418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:39.417677 14418 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:39.418900 14418 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:39.426501 14418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P add86f561fb1454f814759df48c52393: Generated new TSK 0
I20260926 07:39:39.427205 14418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:39.482471 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:39.831663 14434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:39.831931 14434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:39.832021 14434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:39.839075 14434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:39.839196 14434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:39.839308 14434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:39.848691 14434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:39.849854 14434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:39.851154 14434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:39.863883 14439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:39.863920 14442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:39.863850 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:39.864461 14434 server_base.cc:1034] running on GCE node
I20260926 07:39:39.865124 14434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:39.866650 14434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:39.867875 14434 hybrid_clock.cc:648] HybridClock initialized: now 1790408379867807 us; error 53 us; skew 500 ppm
I20260926 07:39:39.868283 14434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:39 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408379, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:39.871400 14434 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:39.872814 14434 webserver.cc:460] Webserver started at http://127.11.9.65:40883/ using document root <none> and password file <none>
I20260926 07:39:39.873543 14434 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:39.873688 14434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:39.873993 14434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:39.876744 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "3053653095f9489ca9641210358e51bb"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "a1f74c6c29757f0ec139b30384861715"
server_key_iv: "181bff0786c9a32dba68c8989fa5267f"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.877565 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "3053653095f9489ca9641210358e51bb"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "a1f74c6c29757f0ec139b30384861715"
server_key_iv: "181bff0786c9a32dba68c8989fa5267f"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.882391 14434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 07:39:39.885588 14449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:39.886852 14434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:39:39.887018 14434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "3053653095f9489ca9641210358e51bb"
format_stamp: "Formatted at 2026-09-26 07:39:39 on dist-test-slave-x492"
server_key: "a1f74c6c29757f0ec139b30384861715"
server_key_iv: "181bff0786c9a32dba68c8989fa5267f"
server_key_version: "encryptionkey@0"
I20260926 07:39:39.887202 14434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:39.907594 14434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:39.911425 14434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:39.911710 14434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:39.912606 14434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:39.913944 14434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:39.914040 14434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:39.914186 14434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:39.914255 14434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:39.946990 14434 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:41217
I20260926 07:39:39.947047 14561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:41217 every 8 connection(s)
I20260926 07:39:39.948242 14434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:39.952256 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14434
I20260926 07:39:39.952484 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:39.953104 11301 external_mini_cluster.cc:1428] Setting key 8bdd6646035f5524eb139929aeac3d3f
Sep 26 07:39:39 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408379, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:39.968441 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:39.950662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:34453 (local address 127.11.9.126:34503)
0926 07:39:39.951074 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:39.951078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:39.952050 (+ 972us) server_negotiation.cc:407] Connection header received
0926 07:39:39.953992 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:39.953996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:39.954112 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:39.954219 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:39.955711 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.956403 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.957978 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.958359 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.958500 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:39.962080 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:39.962085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:39.962088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:39.962118 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:39.965251 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:39.966009 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:39.966017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:39.966023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:39.966167 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:39.966541 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:39.966549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:39.966555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:39.966906 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:39.967074 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:39.967700 (+ 626us) server_negotiation.cc:299] Negotiation successful
0926 07:39:39.967959 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":208,"threads_started":1}
I20260926 07:39:39.968971 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:39.951212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:34503 (local address 127.11.9.65:34453)
0926 07:39:39.951908 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:39.951950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:39.953702 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:39.954432 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:39.954440 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:39.955014 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:39.955557 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:39.955573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.956547 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.956557 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:39.957836 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:39.957848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:39.958586 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:39.958593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:39.958815 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:39.959462 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:39.959488 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:39.961877 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:39.965388 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:39.965394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:39.965405 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:39.965874 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:39.966272 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:39.966275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:39.966277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:39.966438 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:39.967022 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:39.967027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:39.967319 (+ 292us) client_negotiation.cc:769] Sending connection context
0926 07:39:39.967527 (+ 208us) client_negotiation.cc:240] Negotiation successful
0926 07:39:39.967893 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":258,"threads_started":1}
I20260926 07:39:39.970506 14562 heartbeater.cc:347] Connected to a master server at 127.11.9.126:34503
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:39:39.970805 14562 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:39.971483 14562 heartbeater.cc:511] Master 127.11.9.126:34503 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:39:39.973563 14373 ts_manager.cc:194] Registered new tserver with Master: 3053653095f9489ca9641210358e51bb (127.11.9.65:41217)
I20260926 07:39:39.975556 14373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:34453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:39.999553 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:40.313719 14568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:40.313980 14568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:40.314073 14568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:40.320952 14568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:40.321055 14568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:40.321210 14568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:40.330812 14568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:40.332072 14568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:40.333474 14568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:40.346782 14573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:40.347131 14574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:40.347576 14576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:40.347736 14568 server_base.cc:1034] running on GCE node
I20260926 07:39:40.348302 14568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:40.349288 14568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:40.350502 14568 hybrid_clock.cc:648] HybridClock initialized: now 1790408380350449 us; error 50 us; skew 500 ppm
I20260926 07:39:40.350803 14568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:40.354104 14568 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:40.355713 14568 webserver.cc:460] Webserver started at http://127.11.9.66:46267/ using document root <none> and password file <none>
I20260926 07:39:40.356519 14568 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:40.356670 14568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:40.356980 14568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:40.360910 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "30efc6aa7ac845809e735c2e2086d92f"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "36cfe24c9b8b19fb9648fd34a2313eaa"
server_key_iv: "9a8ee02a3c7365d08abba22939f77204"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.361918 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "30efc6aa7ac845809e735c2e2086d92f"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "36cfe24c9b8b19fb9648fd34a2313eaa"
server_key_iv: "9a8ee02a3c7365d08abba22939f77204"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.368453 14568 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 07:39:40.372184 14583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.373575 14568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:40.373739 14568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "30efc6aa7ac845809e735c2e2086d92f"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "36cfe24c9b8b19fb9648fd34a2313eaa"
server_key_iv: "9a8ee02a3c7365d08abba22939f77204"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.373926 14568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:40.391695 14568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:40.395365 14568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:40.395668 14568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:40.396570 14568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:40.397883 14568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:40.397984 14568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.398135 14568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:40.398205 14568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.429703 14568 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:41421
I20260926 07:39:40.429756 14695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:41421 every 8 connection(s)
I20260926 07:39:40.431003 14568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:40.436395 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14568
I20260926 07:39:40.436582 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:40.437091 11301 external_mini_cluster.cc:1428] Setting key 1ce5c866b1a133d1bc62d71e881b1480
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:40.452263 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:40.433399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:46899 (local address 127.11.9.126:34503)
0926 07:39:40.433608 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:40.433615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:40.434783 (+ 1168us) server_negotiation.cc:407] Connection header received
0926 07:39:40.436518 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:40.436525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:40.436621 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:40.436739 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:40.438331 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.439340 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.441891 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.442277 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.442384 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:40.446703 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:40.446708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:40.446711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:40.446739 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:40.449338 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:40.450128 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:40.450133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:40.450137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:40.450263 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:40.450690 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:40.450699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:40.450701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:40.450937 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:40.451080 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:40.451716 (+ 636us) server_negotiation.cc:299] Negotiation successful
0926 07:39:40.451924 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":72}
I20260926 07:39:40.453265 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:40.433940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:34503 (local address 127.11.9.66:46899)
0926 07:39:40.434632 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:40.434675 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:40.436260 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:40.436926 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:40.436933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:40.437512 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:40.438158 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:40.438177 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.439493 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.439498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:40.441761 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:40.441777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.442505 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.442515 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:40.442850 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:40.443554 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:40.443592 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:40.446484 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:40.449471 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:40.449479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:40.449494 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:40.450010 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:40.450384 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:40.450389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:40.450393 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:40.450576 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:40.451057 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:40.451065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:40.451382 (+ 317us) client_negotiation.cc:769] Sending connection context
0926 07:39:40.451626 (+ 244us) client_negotiation.cc:240] Negotiation successful
0926 07:39:40.452127 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":288,"threads_started":1}
I20260926 07:39:40.454638 14696 heartbeater.cc:347] Connected to a master server at 127.11.9.126:34503
I20260926 07:39:40.454968 14696 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:40.455899 14696 heartbeater.cc:511] Master 127.11.9.126:34503 requested a full tablet report, sending...
I20260926 07:39:40.457283 14373 ts_manager.cc:194] Registered new tserver with Master: 30efc6aa7ac845809e735c2e2086d92f (127.11.9.66:41421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
I20260926 07:39:40.458107 14373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:46899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:40.480170 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:40.801936 14701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:40.802210 14701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:40.802397 14701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:40.809572 14701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:40.809695 14701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:40.809881 14701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:40.819362 14701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:34503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:40.820578 14701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:40.822124 14701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:40.834632 14707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:40.834718 14709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:40.835001 14706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:40.835600 14701 server_base.cc:1034] running on GCE node
I20260926 07:39:40.836206 14701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:40.837056 14701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:40.838411 14701 hybrid_clock.cc:648] HybridClock initialized: now 1790408380838322 us; error 78 us; skew 500 ppm
I20260926 07:39:40.838754 14701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:40.841931 14701 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:40.843432 14701 webserver.cc:460] Webserver started at http://127.11.9.67:41035/ using document root <none> and password file <none>
I20260926 07:39:40.844120 14701 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:40.844267 14701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:40.844568 14701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:40.847486 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "bfdaa392ab0a4ab5af66ab7b15f46bc5"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "2d3a5da9a92f9c9f6954a6ea23c965e9"
server_key_iv: "e1157db1d0b44a0488146c11fcb13aa9"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.848255 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "bfdaa392ab0a4ab5af66ab7b15f46bc5"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "2d3a5da9a92f9c9f6954a6ea23c965e9"
server_key_iv: "e1157db1d0b44a0488146c11fcb13aa9"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.853340 14701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 07:39:40.856624 14716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.858016 14701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:40.858234 14701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "bfdaa392ab0a4ab5af66ab7b15f46bc5"
format_stamp: "Formatted at 2026-09-26 07:39:40 on dist-test-slave-x492"
server_key: "2d3a5da9a92f9c9f6954a6ea23c965e9"
server_key_iv: "e1157db1d0b44a0488146c11fcb13aa9"
server_key_version: "encryptionkey@0"
I20260926 07:39:40.858430 14701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:40.883188 14701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:40.887456 14701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:40.887770 14701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:40.888720 14701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:40.890167 14701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:40.890269 14701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.890413 14701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:40.890491 14701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:40.923372 14701 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:43839
I20260926 07:39:40.923419 14828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:43839 every 8 connection(s)
I20260926 07:39:40.924729 14701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:40.927235 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14701
I20260926 07:39:40.927424 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:40.927990 11301 external_mini_cluster.cc:1428] Setting key 071077838305b6b5437e8cc009e34fc3
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:40.944446 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:40.927350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:45389 (local address 127.11.9.126:34503)
0926 07:39:40.928432 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:40.928439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:40.928962 (+ 523us) server_negotiation.cc:407] Connection header received
0926 07:39:40.931076 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:40.931081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:40.931195 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:40.931324 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:40.932841 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.933664 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.935204 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.935600 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.935725 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:40.939048 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:40.939063 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:40.939066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:40.939096 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:40.941444 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:40.942327 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:40.942331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:40.942333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:40.942493 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:40.942966 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:40.942970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:40.942974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:40.943324 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:40.943453 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:40.944056 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 07:39:40.944206 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933}
I20260926 07:39:40.945540 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:40.927932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:34503 (local address 127.11.9.67:45389)
0926 07:39:40.928804 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:40.928857 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:40.930792 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:40.931559 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:40.931568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:40.932071 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:40.932671 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:40.932689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.933818 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.933824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:40.935055 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:40.935066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.935834 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.935841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:40.936078 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:40.936678 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:40.936699 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:40.938855 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:40.941597 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:40.941606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:40.941623 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:40.942216 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:40.942609 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:40.942615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:40.942620 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:40.942857 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:40.943436 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:40.943442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:40.943735 (+ 293us) client_negotiation.cc:769] Sending connection context
0926 07:39:40.943970 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 07:39:40.944345 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":392,"threads_started":1}
I20260926 07:39:40.946761 14829 heartbeater.cc:347] Connected to a master server at 127.11.9.126:34503
I20260926 07:39:40.947068 14829 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:40.947939 14829 heartbeater.cc:511] Master 127.11.9.126:34503 requested a full tablet report, sending...
I20260926 07:39:40.949409 14373 ts_manager.cc:194] Registered new tserver with Master: bfdaa392ab0a4ab5af66ab7b15f46bc5 (127.11.9.67:43839)
I20260926 07:39:40.950163 14373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:45389
I20260926 07:39:40.955132 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:40.979429 14562 heartbeater.cc:503] Master 127.11.9.126:34503 was elected leader, sending a full tablet report...
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 07:39:40 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:41.002089 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:40.987820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.11.9.126:34503)
0926 07:39:40.988055 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:40.988063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:40.988451 (+ 388us) server_negotiation.cc:407] Connection header received
0926 07:39:40.988672 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:40.988676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:40.988737 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:40.988848 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:40.990402 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.991220 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.992514 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:40.992991 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:40.993145 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:40.997141 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:40.997146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:40.997149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:40.997226 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:40.999489 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.000101 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.000106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.000110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.000252 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.000657 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.000664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.000667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.000974 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:41.001273 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:41.001541 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 07:39:41.001745 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260926 07:39:41.002627 14373 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58948
I20260926 07:39:41.004030 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14434
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:41.019726 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:41.005487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.11.9.65:41217)
0926 07:39:41.006104 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:41.006112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:41.006136 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:39:41.006190 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:41.006196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:41.006411 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:41.006534 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:41.007602 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.008581 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.009807 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.010372 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.010467 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:41.014283 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:41.014296 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:41.014313 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:41.014347 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:41.017267 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.017943 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.017949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.017952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.018078 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.018374 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.018377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.018379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.018692 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:41.018903 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:41.019152 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 07:39:41.019334 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":328,"threads_started":1}
W20260926 07:39:41.020991 14557 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59876
W20260926 07:39:41.022166 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14434: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:41.022399 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14434
I20260926 07:39:41.037986 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14568
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:41.053520 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:41.038777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.11.9.66:41421)
0926 07:39:41.039319 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:41.039328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:41.039353 (+ 25us) server_negotiation.cc:407] Connection header received
0926 07:39:41.039410 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:41.039417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:41.039626 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:41.039750 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:41.040883 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.041918 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.043109 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.043683 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.043776 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:41.047603 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:41.047613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:41.047627 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:41.047657 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:41.051017 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.051657 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.051663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.051665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.051773 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.052071 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.052074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.052075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.052380 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:41.052614 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:41.052865 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 07:39:41.053045 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":266,"threads_started":1}
W20260926 07:39:41.054790 14691 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56760
W20260926 07:39:41.055841 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14568: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:41.055995 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14568
I20260926 07:39:41.071715 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14701
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408380, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:41.086687 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:41.072672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.11.9.67:43839)
0926 07:39:41.073404 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:41.073410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:41.073437 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:39:41.073492 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:41.073497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:41.073698 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:41.073816 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:41.074983 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.075990 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.077141 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:41.077750 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:41.077854 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:41.081820 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:41.081830 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:41.081844 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:41.081877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:41.084374 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.084954 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.084960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.084962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.085128 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:41.085464 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:41.085467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:41.085469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:41.085786 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:41.086003 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:41.086221 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 07:39:41.086414 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":371,"threads_started":1}
W20260926 07:39:41.087936 14824 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35346
W20260926 07:39:41.089084 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14701: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:39:41.089282 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14701
I20260926 07:39:41.104518 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14341
I20260926 07:39:41.105634 14403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:41.206732 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14341
2026-09-26T07:39:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3555 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14846](info): set up 2 sockets
Sep 26 07:39:41 dist-test-slave-x492 krb5kdc[14846](info): commencing operation
krb5kdc: starting...
W20260926 07:39:42.281549 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:42 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:42.468477 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35999
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:42.782073 14859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:42.782418 14859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:42.782578 14859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:42.789801 14859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:42.789940 14859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:42.790043 14859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:42.790151 14859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:42.790235 14859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:42.800005 14859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:42.801429 14859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:42.802661 14859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:42.814023 14864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:42.814060 14867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:42.814088 14865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:42.814916 14859 server_base.cc:1034] running on GCE node
I20260926 07:39:42.815593 14859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:42.816869 14859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:42.818091 14859 hybrid_clock.cc:648] HybridClock initialized: now 1790408382818056 us; error 44 us; skew 500 ppm
I20260926 07:39:42.818440 14859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:42 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:42.827538 14859 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:42.829011 14859 webserver.cc:460] Webserver started at http://127.11.9.126:44489/ using document root <none> and password file <none>
I20260926 07:39:42.829743 14859 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:42.829885 14859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:42.830199 14859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:42.832988 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "cffb3479bdaa4027ad38602ddb748404"
format_stamp: "Formatted at 2026-09-26 07:39:42 on dist-test-slave-x492"
server_key: "59a83ca5c39d2ed43ae0bd8ee6526338"
server_key_iv: "0f3ecab033bdeb8d6b07d294856ca29f"
server_key_version: "encryptionkey@0"
I20260926 07:39:42.833819 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "cffb3479bdaa4027ad38602ddb748404"
format_stamp: "Formatted at 2026-09-26 07:39:42 on dist-test-slave-x492"
server_key: "59a83ca5c39d2ed43ae0bd8ee6526338"
server_key_iv: "0f3ecab033bdeb8d6b07d294856ca29f"
server_key_version: "encryptionkey@0"
I20260926 07:39:42.838686 14859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:39:42.842038 14874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:42.843546 14859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:42.843750 14859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "cffb3479bdaa4027ad38602ddb748404"
format_stamp: "Formatted at 2026-09-26 07:39:42 on dist-test-slave-x492"
server_key: "59a83ca5c39d2ed43ae0bd8ee6526338"
server_key_iv: "0f3ecab033bdeb8d6b07d294856ca29f"
server_key_version: "encryptionkey@0"
I20260926 07:39:42.843948 14859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:42.855989 14859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:42.859637 14859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:42.859902 14859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:42.878568 14859 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:35999
I20260926 07:39:42.878602 14925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:35999 every 8 connection(s)
I20260926 07:39:42.880191 14859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:42.883698 14926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:42.884717 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14859
I20260926 07:39:42.884953 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:42.885754 11301 external_mini_cluster.cc:1428] Setting key 7382168fe9b704fe10ca97a4cc784912
I20260926 07:39:42.895915 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404: Bootstrap starting.
Sep 26 07:39:42 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:42.898988 14926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:42.900190 14926 log.cc:824] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:42.903347 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404: No bootstrap required, opened a new log
I20260926 07:39:42.908069 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:42.887740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.11.9.126:35999)
0926 07:39:42.888462 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:42.888473 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:42.888515 (+ 42us) server_negotiation.cc:407] Connection header received
0926 07:39:42.890146 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:42.890171 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:42.890520 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:42.890913 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:42.892310 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:42.893664 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:42.894875 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:42.895508 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:42.895715 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:42.899974 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:42.899981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:42.899993 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:42.900032 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:42.903776 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:42.904408 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:42.904415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:42.904422 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:42.904616 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:42.904968 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:42.904973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:42.904977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:42.905730 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:42.906016 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:42.906328 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 07:39:42.906806 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":322,"threads_started":1}
I20260926 07:39:42.912556 14926 raft_consensus.cc:348] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } }
I20260926 07:39:42.912858 14926 raft_consensus.cc:374] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:42.912930 14926 raft_consensus.cc:729] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffb3479bdaa4027ad38602ddb748404, State: Initialized, Role: FOLLOWER
I20260926 07:39:42.913487 14926 consensus_queue.cc:260] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } }
I20260926 07:39:42.913677 14926 raft_consensus.cc:388] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:42.913769 14926 raft_consensus.cc:482] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:42.913887 14926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:42.916172 14926 raft_consensus.cc:504] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } }
I20260926 07:39:42.916623 14926 leader_election.cc:302] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cffb3479bdaa4027ad38602ddb748404; no voters:
I20260926 07:39:42.917141 14926 leader_election.cc:288] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:42.917332 14931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:42.918787 14931 raft_consensus.cc:686] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [term 1 LEADER]: Becoming Leader. State: Replica: cffb3479bdaa4027ad38602ddb748404, State: Running, Role: LEADER
I20260926 07:39:42.919266 14931 consensus_queue.cc:237] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } }
I20260926 07:39:42.919816 14926 sys_catalog.cc:564] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:42.921988 14932 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cffb3479bdaa4027ad38602ddb748404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } } }
I20260926 07:39:42.921962 14933 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cffb3479bdaa4027ad38602ddb748404. Latest consensus state: current_term: 1 leader_uuid: "cffb3479bdaa4027ad38602ddb748404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffb3479bdaa4027ad38602ddb748404" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35999 } } }
I20260926 07:39:42.922385 14933 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:42.922384 14932 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:42.925833 14937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:42.931277 14937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:42.939296 14937 catalog_manager.cc:1347] Generated new cluster ID: 85b4f903bff5447597c9998808acac9f
I20260926 07:39:42.939396 14937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:42.952462 14937 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:42.954319 14937 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:42.967525 14937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cffb3479bdaa4027ad38602ddb748404: Generated new TSK 0
I20260926 07:39:42.968315 14937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:43.018743 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:43.335084 14952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:43.335350 14952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:43.335443 14952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:43.342540 14952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:43.342636 14952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:43.342751 14952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:43.352346 14952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:43.353585 14952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:43.354818 14952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:43.367612 14960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:43.367689 14957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:43.367717 14958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:43.368434 14952 server_base.cc:1034] running on GCE node
I20260926 07:39:43.369024 14952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:43.370014 14952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:43.371214 14952 hybrid_clock.cc:648] HybridClock initialized: now 1790408383371173 us; error 32 us; skew 500 ppm
I20260926 07:39:43.371469 14952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:43 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:43.374413 14952 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:43.375806 14952 webserver.cc:460] Webserver started at http://127.11.9.65:38583/ using document root <none> and password file <none>
I20260926 07:39:43.376422 14952 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:43.376518 14952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:43.376767 14952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:43.379585 14952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "266efc1d712b4526b2672a039107d263"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "6aae23de844d901c78d8e131e853c0a2"
server_key_iv: "a00471d19840f9e48bc57c5d7169627a"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.380292 14952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "266efc1d712b4526b2672a039107d263"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "6aae23de844d901c78d8e131e853c0a2"
server_key_iv: "a00471d19840f9e48bc57c5d7169627a"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.385074 14952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260926 07:39:43.388315 14967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.389627 14952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 07:39:43.389812 14952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "266efc1d712b4526b2672a039107d263"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "6aae23de844d901c78d8e131e853c0a2"
server_key_iv: "a00471d19840f9e48bc57c5d7169627a"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.390013 14952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:43.411108 14952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:43.414970 14952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:43.415259 14952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:43.416237 14952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:43.417532 14952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:43.417637 14952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.417784 14952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:43.417856 14952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.450496 14952 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:32831
I20260926 07:39:43.450551 15079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:32831 every 8 connection(s)
I20260926 07:39:43.451756 14952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:43.456831 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 14952
I20260926 07:39:43.457057 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:43.457763 11301 external_mini_cluster.cc:1428] Setting key 408409f4ae67ba3652f2cb1bc279ea88
Sep 26 07:39:43 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:43.472361 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:43.454290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38495 (local address 127.11.9.126:35999)
0926 07:39:43.454730 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:43.454734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:43.455615 (+ 881us) server_negotiation.cc:407] Connection header received
0926 07:39:43.457568 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:43.457575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:43.457663 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:43.457778 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:43.459393 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.460122 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.461706 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.462115 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.462248 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:43.466100 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:43.466105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:43.466108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:43.466138 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:43.468976 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:43.469994 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:43.470019 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:43.470025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:43.470161 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:43.470676 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:43.470679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:43.470682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:43.470938 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:43.471048 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:43.471945 (+ 897us) server_negotiation.cc:299] Negotiation successful
0926 07:39:43.472096 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":243,"threads_started":1}
I20260926 07:39:43.474115 15083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:43.454731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35999 (local address 127.11.9.65:38495)
0926 07:39:43.455457 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:43.455494 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:43.457297 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:43.458019 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:43.458027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:43.458669 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:43.459222 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:43.459241 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.460284 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.460288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:43.461563 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:43.461575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.462331 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.462340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:43.462577 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:43.463261 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:43.463282 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:43.465891 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:43.469117 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:43.469127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:43.469146 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:43.469846 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:43.470278 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:43.470286 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:43.470293 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:43.470541 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:43.471071 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:43.471081 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:43.471521 (+ 440us) client_negotiation.cc:769] Sending connection context
0926 07:39:43.471855 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 07:39:43.472425 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":293,"threads_started":1}
I20260926 07:39:43.475692 15080 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35999
I20260926 07:39:43.475984 15080 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:39:43.476686 15080 heartbeater.cc:511] Master 127.11.9.126:35999 requested a full tablet report, sending...
I20260926 07:39:43.478689 14891 ts_manager.cc:194] Registered new tserver with Master: 266efc1d712b4526b2672a039107d263 (127.11.9.65:32831)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:39:43.480162 14891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:38495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:43.500437 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:43.808005 15086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:43.808269 15086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:43.808379 15086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:43.815598 15086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:43.815691 15086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:43.815900 15086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:43.825129 15086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:43.826444 15086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:43.827762 15086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:43.840219 15092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:43.840382 15094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:43.840669 15091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:43.841032 15086 server_base.cc:1034] running on GCE node
I20260926 07:39:43.841661 15086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:43.842545 15086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:43.843770 15086 hybrid_clock.cc:648] HybridClock initialized: now 1790408383843726 us; error 57 us; skew 500 ppm
I20260926 07:39:43.844081 15086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:43 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:43.847295 15086 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:43.848776 15086 webserver.cc:460] Webserver started at http://127.11.9.66:38211/ using document root <none> and password file <none>
I20260926 07:39:43.849521 15086 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:43.849663 15086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:43.849964 15086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:43.852715 15086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "e6d920de2cfd4264bc2fd871a68e9ffd"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "87d623974381245d5aa98bcdc292ce07"
server_key_iv: "b3bbdaff8d62756f5bcb52389c8df5f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.853538 15086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "e6d920de2cfd4264bc2fd871a68e9ffd"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "87d623974381245d5aa98bcdc292ce07"
server_key_iv: "b3bbdaff8d62756f5bcb52389c8df5f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.858342 15086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 07:39:43.861670 15101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.862859 15086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:43.863042 15086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "e6d920de2cfd4264bc2fd871a68e9ffd"
format_stamp: "Formatted at 2026-09-26 07:39:43 on dist-test-slave-x492"
server_key: "87d623974381245d5aa98bcdc292ce07"
server_key_iv: "b3bbdaff8d62756f5bcb52389c8df5f3"
server_key_version: "encryptionkey@0"
I20260926 07:39:43.863257 15086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:43.879127 15086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:43.884398 15086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:43.884681 15086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:43.885684 15086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:43.886979 15086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:43.887079 15086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.887221 15086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:43.887291 15086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:43.920863 15086 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45005
I20260926 07:39:43.920898 15213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45005 every 8 connection(s)
I20260926 07:39:43.922185 15086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:43.927318 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15086
I20260926 07:39:43.927526 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:43.927963 11301 external_mini_cluster.cc:1428] Setting key adfc09bd69ab0e777083a1e7e8b8e42d
Sep 26 07:39:43 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408383, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:43.942137 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:43.924572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:59857 (local address 127.11.9.126:35999)
0926 07:39:43.924735 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:43.924739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:43.925941 (+ 1202us) server_negotiation.cc:407] Connection header received
0926 07:39:43.927895 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:43.927900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:43.927988 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:43.928094 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:43.929794 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.930507 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.932013 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.932404 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.932534 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:43.936452 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:43.936457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:43.936461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:43.936490 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:43.939001 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:43.939880 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:43.939884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:43.939886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:43.940008 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:43.940478 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:43.940481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:43.940483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:43.940709 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:43.940819 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:43.941710 (+ 891us) server_negotiation.cc:299] Negotiation successful
0926 07:39:43.941871 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 07:39:43.943974 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:43.924995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35999 (local address 127.11.9.66:59857)
0926 07:39:43.925736 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:43.925799 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:43.927588 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:43.928337 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:43.928344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:43.928827 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:43.929624 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:43.929656 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.930656 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.930661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:43.931865 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:43.931876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:43.932618 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:43.932627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:43.932887 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:43.933574 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:43.933595 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:43.936233 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:43.939140 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:43.939151 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:43.939170 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:43.939765 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:43.940109 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:43.940117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:43.940123 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:43.940369 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:43.940843 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:43.940853 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:43.941328 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 07:39:43.941625 (+ 297us) client_negotiation.cc:240] Negotiation successful
0926 07:39:43.942205 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":340,"threads_started":1}
I20260926 07:39:43.945515 15214 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35999
I20260926 07:39:43.945844 15214 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:43.946743 15214 heartbeater.cc:511] Master 127.11.9.126:35999 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:43.948272 14891 ts_manager.cc:194] Registered new tserver with Master: e6d920de2cfd4264bc2fd871a68e9ffd (127.11.9.66:45005)
I20260926 07:39:43.949103 14891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:59857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:43.971596 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:44.286574 15219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:44.286829 15219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:44.286990 15219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:44.293922 15219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:44.294044 15219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:44.294235 15219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:44.303649 15219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:44.304853 15219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:44.306090 15219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:44.318917 15224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:44.318938 15227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:44.318907 15225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:44.319746 15219 server_base.cc:1034] running on GCE node
I20260926 07:39:44.320312 15219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:44.321285 15219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:44.322505 15219 hybrid_clock.cc:648] HybridClock initialized: now 1790408384322454 us; error 62 us; skew 500 ppm
I20260926 07:39:44.322814 15219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:44.325910 15219 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:44.327453 15219 webserver.cc:460] Webserver started at http://127.11.9.67:36515/ using document root <none> and password file <none>
I20260926 07:39:44.328066 15219 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:44.328162 15219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:44.328415 15219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:44.331132 15219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "5cd554bf29d24e42984d41232f7f8bc2"
format_stamp: "Formatted at 2026-09-26 07:39:44 on dist-test-slave-x492"
server_key: "177bdf6293dae7c22b5298f4825dd5a4"
server_key_iv: "979225995212d40536b376b845c87b58"
server_key_version: "encryptionkey@0"
I20260926 07:39:44.331817 15219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "5cd554bf29d24e42984d41232f7f8bc2"
format_stamp: "Formatted at 2026-09-26 07:39:44 on dist-test-slave-x492"
server_key: "177bdf6293dae7c22b5298f4825dd5a4"
server_key_iv: "979225995212d40536b376b845c87b58"
server_key_version: "encryptionkey@0"
I20260926 07:39:44.336519 15219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:39:44.339670 15234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:44.340848 15219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:44.340968 15219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "5cd554bf29d24e42984d41232f7f8bc2"
format_stamp: "Formatted at 2026-09-26 07:39:44 on dist-test-slave-x492"
server_key: "177bdf6293dae7c22b5298f4825dd5a4"
server_key_iv: "979225995212d40536b376b845c87b58"
server_key_version: "encryptionkey@0"
I20260926 07:39:44.341121 15219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:44.370590 15219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:44.374481 15219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:44.374759 15219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:44.375627 15219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:44.376820 15219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:44.376922 15219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:44.377080 15219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:44.377183 15219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:44.409309 15219 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:37315
I20260926 07:39:44.409329 15346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:37315 every 8 connection(s)
I20260926 07:39:44.410593 15219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:44.420923 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15219
I20260926 07:39:44.421233 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:44.421669 11301 external_mini_cluster.cc:1428] Setting key 3d51f548b9f0cde80178b2dea877ff8e
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:44.429502 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.413081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:39293 (local address 127.11.9.126:35999)
0926 07:39:44.413319 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.413325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.414528 (+ 1203us) server_negotiation.cc:407] Connection header received
0926 07:39:44.416198 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.416202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.416265 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.416351 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.417850 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.418553 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.420174 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.420679 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.420835 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.424498 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.424503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.424506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.424534 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.427030 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.427696 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.427700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.427702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.427809 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.428154 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.428157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.428159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.428388 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.428500 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.429049 (+ 549us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.429264 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 07:39:44.430449 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.413653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35999 (local address 127.11.9.67:39293)
0926 07:39:44.414360 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:44.414403 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:44.415966 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:44.416534 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:44.416542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:44.417014 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:44.417712 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:44.417729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.418713 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.418717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:44.420058 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:44.420069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.420940 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.420946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:44.421343 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.422073 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:44.422103 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:44.424304 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:44.427151 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:44.427157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:44.427168 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:44.427561 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:44.427898 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:44.427901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:44.427904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:44.428061 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:44.428494 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:44.428499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:44.428765 (+ 266us) client_negotiation.cc:769] Sending connection context
0926 07:39:44.428971 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 07:39:44.429367 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":269,"threads_started":1}
I20260926 07:39:44.431787 15347 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35999
I20260926 07:39:44.432039 15347 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:44.432653 15347 heartbeater.cc:511] Master 127.11.9.126:35999 requested a full tablet report, sending...
I20260926 07:39:44.433961 14891 ts_manager.cc:194] Registered new tserver with Master: 5cd554bf29d24e42984d41232f7f8bc2 (127.11.9.67:37315)
I20260926 07:39:44.434666 14891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:39293
I20260926 07:39:44.436704 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:44.482782 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.468727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44508 (local address 127.11.9.65:32831)
0926 07:39:44.469321 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.469334 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.469361 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:39:44.469418 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.469425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.469623 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.469745 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.471091 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.472185 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.473466 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.473996 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.474088 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.477608 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.477618 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.477632 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.477664 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.480382 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.480908 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.480915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.480917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.481028 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.481414 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.481418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.481420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.481719 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.481929 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.482135 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.482310 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":282,"threads_started":1}
I20260926 07:39:44.482909 15080 heartbeater.cc:503] Master 127.11.9.126:35999 was elected leader, sending a full tablet report...
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:44.500016 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.486988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33870 (local address 127.11.9.126:35999)
0926 07:39:44.487192 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.487199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.487421 (+ 222us) server_negotiation.cc:407] Connection header received
0926 07:39:44.487587 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.487592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.487661 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.487762 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.488892 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.489696 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.490905 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.491325 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.491476 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.495199 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.495204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.495207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.495236 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.497490 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.498104 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.498111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.498114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.498255 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.498620 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.498625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.498629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.498936 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.499185 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.499492 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.499699 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 07:39:44.500577 14891 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33870
I20260926 07:39:44.501621 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14952
I20260926 07:39:44.514348 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.502571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.11.9.65:32831)
0926 07:39:44.502819 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.502828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.503276 (+ 448us) server_negotiation.cc:407] Connection header received
0926 07:39:44.503471 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.503475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.503561 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.503663 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.504800 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.505527 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.506735 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.507184 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.507322 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.509508 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.509513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.509516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.509545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.511869 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.512479 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.512484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.512487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.512604 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.512959 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.512964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.512967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.513277 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.513509 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.513809 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.514028 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:39:44.514914 15075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:44.599714 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14952
I20260926 07:39:44.615732 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15086
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:44.630877 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.616575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.11.9.66:45005)
0926 07:39:44.617082 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.617089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.617113 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:39:44.617225 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.617232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.617450 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.617623 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.618761 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.619746 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.621089 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.621756 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.621895 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.625823 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.625833 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.625849 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.625881 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.628499 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.629072 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.629078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.629080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.629240 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.629549 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.629552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.629554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.629850 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.630053 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.630306 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.630528 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":254,"threads_started":1}
I20260926 07:39:44.632138 15209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:44.713027 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15086
I20260926 07:39:44.728929 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15219
Sep 26 07:39:44 dist-test-slave-x492 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:44.744352 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:44.729964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.11.9.67:37315)
0926 07:39:44.730513 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:44.730520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:44.730547 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:39:44.730604 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:44.730610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:44.730840 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:44.731013 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:44.732265 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.733337 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.734640 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:44.735183 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:44.735320 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:44.739441 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:44.739450 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:44.739463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:44.739493 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:44.742002 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.742617 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.742623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.742626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.742744 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:44.743057 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:44.743060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:44.743062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:44.743368 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:44.743576 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:44.743853 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 07:39:44.744042 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":275,"threads_started":1}
I20260926 07:39:44.745709 15342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:44.827615 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15219
I20260926 07:39:44.842953 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 14859
I20260926 07:39:44.844105 14921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:44.947542 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 14859
2026-09-26T07:39:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3740 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:45 dist-test-slave-x492 krb5kdc[15366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:45 dist-test-slave-x492 krb5kdc[15366](info): set up 2 sockets
Sep 26 07:39:45 dist-test-slave-x492 krb5kdc[15366](info): commencing operation
krb5kdc: starting...
W20260926 07:39:46.049618 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:46 dist-test-slave-x492 krb5kdc[15366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:46.238804 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35865
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:46.556089 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:46.556420 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:46.556610 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:46.563638 15379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:46.563788 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:46.563872 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:46.563977 15379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:46.564049 15379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:46.573810 15379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:46.575105 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:46.576354 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:46.587841 15384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:46.587795 15385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:46.587747 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:46.588694 15379 server_base.cc:1034] running on GCE node
I20260926 07:39:46.589521 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:46.590857 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:46.592070 15379 hybrid_clock.cc:648] HybridClock initialized: now 1790408386592021 us; error 44 us; skew 500 ppm
I20260926 07:39:46.592463 15379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:46 dist-test-slave-x492 krb5kdc[15366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:46.595510 15379 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:46.596658 15379 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:46.596767 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:46.597091 15379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:46.599932 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7"
format_stamp: "Formatted at 2026-09-26 07:39:46 on dist-test-slave-x492"
server_key: "53b575776fb589c490f2b20632fc408b"
server_key_iv: "b8b548ec22eecb4c405994f826f3e913"
server_key_version: "encryptionkey@0"
I20260926 07:39:46.600701 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7"
format_stamp: "Formatted at 2026-09-26 07:39:46 on dist-test-slave-x492"
server_key: "53b575776fb589c490f2b20632fc408b"
server_key_iv: "b8b548ec22eecb4c405994f826f3e913"
server_key_version: "encryptionkey@0"
I20260926 07:39:46.605556 15379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260926 07:39:46.608770 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:46.610052 15379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:46.610224 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7"
format_stamp: "Formatted at 2026-09-26 07:39:46 on dist-test-slave-x492"
server_key: "53b575776fb589c490f2b20632fc408b"
server_key_iv: "b8b548ec22eecb4c405994f826f3e913"
server_key_version: "encryptionkey@0"
I20260926 07:39:46.610440 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:46.644026 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:46.647927 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:46.648202 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:46.667099 15379 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:35865
I20260926 07:39:46.667119 15444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:35865 every 8 connection(s)
I20260926 07:39:46.668105 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:46.671696 15445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:46.675838 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15379
I20260926 07:39:46.676075 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:46.676865 11301 external_mini_cluster.cc:1428] Setting key 799f5f5d459fa3eebad8982c18d66aa1
I20260926 07:39:46.683393 15445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: Bootstrap starting.
I20260926 07:39:46.687062 15445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:46.688326 15445 log.cc:824] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:46.691185 15445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: No bootstrap required, opened a new log
Sep 26 07:39:46 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:46.700145 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:46.679049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.11.9.126:35865)
0926 07:39:46.679791 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:46.679803 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:46.679851 (+ 48us) server_negotiation.cc:407] Connection header received
0926 07:39:46.681788 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:46.681811 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:46.682179 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:46.682489 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:46.684298 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:46.685713 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:46.687667 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:46.688394 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:46.688593 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:46.692896 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:46.692904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:46.692918 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:46.692960 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:46.696024 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:46.696861 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:46.696867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:46.696877 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:46.697055 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:46.697493 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:46.697496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:46.697498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:46.698039 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:46.698243 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:46.698710 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 07:39:46.699099 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":364,"threads_started":1}
I20260926 07:39:46.700562 15445 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } }
I20260926 07:39:46.700960 15445 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:46.701078 15445 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6e0e1d6c984c04a4a33ab0e740dbd7, State: Initialized, Role: FOLLOWER
I20260926 07:39:46.701695 15445 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } }
I20260926 07:39:46.701982 15445 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:46.702126 15445 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:46.702327 15445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:46.704795 15445 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } }
I20260926 07:39:46.705416 15445 leader_election.cc:302] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6e0e1d6c984c04a4a33ab0e740dbd7; no voters:
I20260926 07:39:46.706005 15445 leader_election.cc:288] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:46.706139 15450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:46.706802 15450 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [term 1 LEADER]: Becoming Leader. State: Replica: ce6e0e1d6c984c04a4a33ab0e740dbd7, State: Running, Role: LEADER
I20260926 07:39:46.707286 15450 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } }
I20260926 07:39:46.708063 15445 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:46.710165 15451 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } } }
I20260926 07:39:46.710304 15452 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6e0e1d6c984c04a4a33ab0e740dbd7. Latest consensus state: current_term: 1 leader_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6e0e1d6c984c04a4a33ab0e740dbd7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35865 } } }
I20260926 07:39:46.710729 15451 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:46.711490 15452 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:46.712209 15457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:46.717514 15457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:46.726873 15457 catalog_manager.cc:1347] Generated new cluster ID: a5b6f9151fd94b6a83569f35169fe1d5
I20260926 07:39:46.726981 15457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:46.745245 15457 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:46.746695 15457 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:46.759086 15457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: Generated new TSK 0
I20260926 07:39:46.759852 15457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:46.815228 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:47.143018 15471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:47.143276 15471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:47.143371 15471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:47.150769 15471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:47.150869 15471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:47.151023 15471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:47.160467 15471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:47.161839 15471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:47.163159 15471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:47.178536 15477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:47.178759 15479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:47.178961 15476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:47.179723 15471 server_base.cc:1034] running on GCE node
I20260926 07:39:47.180316 15471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:47.181337 15471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:47.182539 15471 hybrid_clock.cc:648] HybridClock initialized: now 1790408387182489 us; error 45 us; skew 500 ppm
I20260926 07:39:47.182832 15471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:47 dist-test-slave-x492 krb5kdc[15366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:47.185915 15471 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:47.187013 15471 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:47.187127 15471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:47.187458 15471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:47.190596 15471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "eeca62c2bf4a41c3aed5449f58b13072"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "03c35fb3ce9df5a60c8594b2254e1991"
server_key_iv: "af4eb230785a12e2cc459f53faa4670b"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.191409 15471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "eeca62c2bf4a41c3aed5449f58b13072"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "03c35fb3ce9df5a60c8594b2254e1991"
server_key_iv: "af4eb230785a12e2cc459f53faa4670b"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.196296 15471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 07:39:47.199609 15485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.200896 15471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:47.201073 15471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "eeca62c2bf4a41c3aed5449f58b13072"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "03c35fb3ce9df5a60c8594b2254e1991"
server_key_iv: "af4eb230785a12e2cc459f53faa4670b"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.201344 15471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:47.216678 15471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:47.220419 15471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:47.220690 15471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:47.221647 15471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:47.222904 15471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:47.223021 15471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.223142 15471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:47.223242 15471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.255693 15471 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:36711
I20260926 07:39:47.255779 15597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:36711 every 8 connection(s)
I20260926 07:39:47.256551 15471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:47.265604 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15471
I20260926 07:39:47.265825 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:47.266615 11301 external_mini_cluster.cc:1428] Setting key 29e97599e4b7df8c26afbe980f6433bb
Sep 26 07:39:47 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:47.277684 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:47.258949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:60977 (local address 127.11.9.126:35865)
0926 07:39:47.259470 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:47.259474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:47.260403 (+ 929us) server_negotiation.cc:407] Connection header received
0926 07:39:47.262204 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:47.262209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:47.262305 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:47.262422 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:47.263982 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.264743 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.267130 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.267620 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.267738 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:47.271266 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:47.271275 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:47.271282 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:47.271325 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:47.274570 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:47.275509 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:47.275513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:47.275515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:47.275624 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:47.276067 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:47.276070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:47.276072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:47.276316 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:47.276418 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:47.277302 (+ 884us) server_negotiation.cc:299] Negotiation successful
0926 07:39:47.277449 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":237,"threads_started":1}
I20260926 07:39:47.279182 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:47.259552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35865 (local address 127.11.9.65:60977)
0926 07:39:47.260223 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:47.260262 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:47.261952 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:47.262636 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:47.262643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:47.263142 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:47.263666 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:47.263682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.264907 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.264913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:47.266982 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:47.266997 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.267874 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.267881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:47.268144 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:47.268911 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:47.268937 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:47.271064 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:47.274721 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:47.274751 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:47.274769 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:47.275392 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:47.275721 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:47.275734 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:47.275741 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:47.275962 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:47.276434 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:47.276445 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:47.276874 (+ 429us) client_negotiation.cc:769] Sending connection context
0926 07:39:47.277200 (+ 326us) client_negotiation.cc:240] Negotiation successful
0926 07:39:47.277785 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":268,"threads_started":1}
I20260926 07:39:47.280676 15598 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35865
I20260926 07:39:47.280961 15598 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:47.281654 15598 heartbeater.cc:511] Master 127.11.9.126:35865 requested a full tablet report, sending...
I20260926 07:39:47.283684 15410 ts_manager.cc:194] Registered new tserver with Master: eeca62c2bf4a41c3aed5449f58b13072 (127.11.9.65:36711)
I20260926 07:39:47.285043 15410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:60977
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:47.310528 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:47.621088 15604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:47.621505 15604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:47.621605 15604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:47.628620 15604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:47.628713 15604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:47.628823 15604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:47.638176 15604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:47.639354 15604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:47.640563 15604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:47.652616 15609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:47.652661 15612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:47.652632 15610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:47.654029 15604 server_base.cc:1034] running on GCE node
I20260926 07:39:47.654589 15604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:47.655441 15604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:47.656677 15604 hybrid_clock.cc:648] HybridClock initialized: now 1790408387656633 us; error 55 us; skew 500 ppm
I20260926 07:39:47.656994 15604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:47 dist-test-slave-x492 krb5kdc[15366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:47.660063 15604 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:47.661228 15604 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:47.661367 15604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:47.661665 15604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:47.664351 15604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "0bf01ca302fe4b13b229c60736da97e5"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "658c3482005aaeb180fe662fc5057097"
server_key_iv: "08e4ab8b4e5ea5d1c96d8f058eee4e7d"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.665354 15604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "0bf01ca302fe4b13b229c60736da97e5"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "658c3482005aaeb180fe662fc5057097"
server_key_iv: "08e4ab8b4e5ea5d1c96d8f058eee4e7d"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.670317 15604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 07:39:47.673758 15618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.675037 15604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 07:39:47.675233 15604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "0bf01ca302fe4b13b229c60736da97e5"
format_stamp: "Formatted at 2026-09-26 07:39:47 on dist-test-slave-x492"
server_key: "658c3482005aaeb180fe662fc5057097"
server_key_iv: "08e4ab8b4e5ea5d1c96d8f058eee4e7d"
server_key_version: "encryptionkey@0"
I20260926 07:39:47.675485 15604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:47.700049 15604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:47.704296 15604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:47.704639 15604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:47.705637 15604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:47.706887 15604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:47.707005 15604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.707123 15604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:47.707207 15604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:47.739631 15604 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:33185
I20260926 07:39:47.739686 15730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:33185 every 8 connection(s)
I20260926 07:39:47.740492 15604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:47.746644 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15604
I20260926 07:39:47.746847 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:47.747293 11301 external_mini_cluster.cc:1428] Setting key 4fa61ea82a70849baad44c05ef2f5abd
Sep 26 07:39:47 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408387, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:47.760309 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:47.743021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:49325 (local address 127.11.9.126:35865)
0926 07:39:47.743194 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:47.743198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:47.744256 (+ 1058us) server_negotiation.cc:407] Connection header received
0926 07:39:47.746100 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:47.746104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:47.746196 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:47.746303 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:47.747880 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.748728 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.750458 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.751233 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.751373 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:47.754845 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:47.754850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:47.754854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:47.754886 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:47.757475 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:47.758170 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:47.758176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:47.758181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:47.758302 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:47.758724 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:47.758729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:47.758732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:47.758971 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:47.759121 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:47.759740 (+ 619us) server_negotiation.cc:299] Negotiation successful
0926 07:39:47.759968 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:39:47.761412 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:47.743445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35865 (local address 127.11.9.66:49325)
0926 07:39:47.744104 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:47.744141 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:47.745839 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:47.746531 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:47.746539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:47.747127 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:47.747717 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:47.747734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.748956 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.748962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:47.750306 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:47.750318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:47.751426 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:47.751431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:47.751615 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:47.752285 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:47.752306 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:47.754633 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:47.757626 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:47.757635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:47.757651 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:47.758045 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:47.758419 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:47.758425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:47.758429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:47.758608 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:47.759095 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:47.759103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:47.759410 (+ 307us) client_negotiation.cc:769] Sending connection context
0926 07:39:47.759653 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 07:39:47.760189 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":266,"threads_started":1}
I20260926 07:39:47.762863 15731 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35865
I20260926 07:39:47.763209 15731 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:47.763993 15731 heartbeater.cc:511] Master 127.11.9.126:35865 requested a full tablet report, sending...
I20260926 07:39:47.765590 15410 ts_manager.cc:194] Registered new tserver with Master: 0bf01ca302fe4b13b229c60736da97e5 (127.11.9.66:33185)
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:39:47.766386 15410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:49325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:47.793154 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:48.114241 15736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:48.114498 15736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:48.114593 15736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:48.122013 15736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:48.122112 15736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:48.122225 15736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:48.131527 15736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:35865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:48.132854 15736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:48.134127 15736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:48.146754 15741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:48.146860 15744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:48.146905 15742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:48.147712 15736 server_base.cc:1034] running on GCE node
I20260926 07:39:48.148263 15736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:48.149106 15736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:48.150465 15736 hybrid_clock.cc:648] HybridClock initialized: now 1790408388150416 us; error 44 us; skew 500 ppm
I20260926 07:39:48.150753 15736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:48 dist-test-slave-x492 krb5kdc[15366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408388, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:48.156728 15736 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:48.157872 15736 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:48.158008 15736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:48.158326 15736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:48.161252 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "a1f34046580e4e6f88e60b168a7ef5a2"
format_stamp: "Formatted at 2026-09-26 07:39:48 on dist-test-slave-x492"
server_key: "0001114fd11474cdb81178296e368e7d"
server_key_iv: "45c313ef3bb8a1625831f48565532d8f"
server_key_version: "encryptionkey@0"
I20260926 07:39:48.162101 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "a1f34046580e4e6f88e60b168a7ef5a2"
format_stamp: "Formatted at 2026-09-26 07:39:48 on dist-test-slave-x492"
server_key: "0001114fd11474cdb81178296e368e7d"
server_key_iv: "45c313ef3bb8a1625831f48565532d8f"
server_key_version: "encryptionkey@0"
I20260926 07:39:48.167817 15736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:48.171187 15750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:48.172478 15736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:39:48.172653 15736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "a1f34046580e4e6f88e60b168a7ef5a2"
format_stamp: "Formatted at 2026-09-26 07:39:48 on dist-test-slave-x492"
server_key: "0001114fd11474cdb81178296e368e7d"
server_key_iv: "45c313ef3bb8a1625831f48565532d8f"
server_key_version: "encryptionkey@0"
I20260926 07:39:48.172868 15736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:48.195339 15736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:48.199172 15736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:48.199447 15736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:48.200366 15736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:48.201687 15736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:48.201808 15736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:48.201934 15736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:48.202021 15736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:48.234285 15736 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:37569
I20260926 07:39:48.234321 15862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:37569 every 8 connection(s)
I20260926 07:39:48.235162 15736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:48.237907 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15736
I20260926 07:39:48.238104 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:48.238704 11301 external_mini_cluster.cc:1428] Setting key 2a2b3b65fb3e5ee7923b5203441ca457
Sep 26 07:39:48 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408388, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:48.258518 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.237603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:55073 (local address 127.11.9.126:35865)
0926 07:39:48.237787 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.237791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.239233 (+ 1442us) server_negotiation.cc:407] Connection header received
0926 07:39:48.242114 (+ 2881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.242118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.242232 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.242374 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:48.244457 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.245796 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.247939 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.248573 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.248756 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.252880 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:48.252885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:48.252888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:48.252918 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:48.255559 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.256297 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.256305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.256309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.256441 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.256838 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.256844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.256851 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.257105 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:48.257319 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.257972 (+ 653us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.258172 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:39:48.259462 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.238261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35865 (local address 127.11.9.67:55073)
0926 07:39:48.239073 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.239112 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.241759 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.242686 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.242698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.243421 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:48.244198 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.244225 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.246012 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.246017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.247765 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.247784 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.248894 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.248908 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.249296 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.249970 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:48.249999 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:48.252671 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:48.255692 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:48.255701 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:48.255715 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:48.256145 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:48.256549 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:48.256555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:48.256558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:48.256729 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:48.257304 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:48.257312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:48.257613 (+ 301us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.257885 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.258375 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":337,"threads_started":1}
I20260926 07:39:48.260748 15863 heartbeater.cc:347] Connected to a master server at 127.11.9.126:35865
I20260926 07:39:48.261030 15863 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:48.261659 15863 heartbeater.cc:511] Master 127.11.9.126:35865 requested a full tablet report, sending...
I20260926 07:39:48.262979 15410 ts_manager.cc:194] Registered new tserver with Master: a1f34046580e4e6f88e60b168a7ef5a2 (127.11.9.67:37569)
I20260926 07:39:48.263746 15410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:55073
I20260926 07:39:48.266098 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:48.282371 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.270398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.11.9.126:35865)
0926 07:39:48.270563 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.270571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.270994 (+ 423us) server_negotiation.cc:407] Connection header received
0926 07:39:48.271184 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.271189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.271267 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.271367 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:48.272700 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.273460 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.274683 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.275115 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.275242 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.277550 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:48.277556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:48.277560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:48.277597 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:48.279844 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.280471 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.280476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.280480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.280612 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.280971 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.280977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.280980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.281330 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:48.281608 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.281879 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.282071 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 07:39:48.287940 15598 heartbeater.cc:503] Master 127.11.9.126:35865 was elected leader, sending a full tablet report...
I20260926 07:39:48.286067 15410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:48.293262 15410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:48.316632 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.304173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:37569 (local address 127.0.0.1:54264)
0926 07:39:48.306184 (+ 2011us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.306225 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.306380 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.308186 (+ 1806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.308193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.308222 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:48.308667 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.308680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.311074 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.311080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.313664 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.313679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.315720 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.315726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.315921 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.315972 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.316090 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.316270 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1793,"thread_start_us":2317,"threads_started":1}
I20260926 07:39:48.318076 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.304323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:33185 (local address 127.0.0.1:60990)
0926 07:39:48.306800 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.306816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.306914 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.308774 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.308780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.308809 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:48.309262 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.309276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.312091 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.312096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.314227 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.314240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.317313 (+ 3073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.317319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.317510 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.317541 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.317614 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.317731 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2323,"thread_start_us":1513,"threads_started":1}
I20260926 07:39:48.318071 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.304548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.11.9.67:37569)
0926 07:39:48.307460 (+ 2912us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.307468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.307498 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:39:48.307594 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.307602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.307947 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.308899 (+ 952us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:48.309305 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.310913 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.313872 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.315500 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.317244 (+ 1744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.317488 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.317556 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.317722 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2750,"thread_start_us":2583,"threads_started":1}
I20260926 07:39:48.319123 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.304903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.11.9.66:33185)
0926 07:39:48.306960 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.306970 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.306996 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:39:48.308082 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.308092 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.308318 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.308525 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:48.309465 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.311937 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.314472 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.316808 (+ 2336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.316930 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.317139 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.318513 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.318722 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1909,"thread_start_us":3275,"threads_started":1}
I20260926 07:39:48.323469 15798 tablet_service.cc:1467] Processing CreateTablet for tablet 65a663298a644e35be294363195b526e (DEFAULT_TABLE table=test-table [id=d981356bbed14b2391f2f05357f27538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:48.323520 15666 tablet_service.cc:1467] Processing CreateTablet for tablet 65a663298a644e35be294363195b526e (DEFAULT_TABLE table=test-table [id=d981356bbed14b2391f2f05357f27538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:48.324719 15666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a663298a644e35be294363195b526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:48.324743 15798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a663298a644e35be294363195b526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:48.326953 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.306955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38922)
0926 07:39:48.316957 (+ 10002us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.316974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.317073 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.320489 (+ 3416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.320501 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.320524 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:48.320678 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.320691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.322584 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.322592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.324341 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.324356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.326225 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.326230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.326396 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.326437 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.326535 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.326696 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9898,"thread_start_us":12981,"threads_started":1}
I20260926 07:39:48.327045 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.308050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.11.9.65:36711)
0926 07:39:48.319861 (+ 11811us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.319869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.319899 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:39:48.320008 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.320018 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.320279 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.320520 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:48.320836 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.322410 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.324510 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.326012 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.326187 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.326392 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.326591 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.326736 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11632,"thread_start_us":11460,"threads_started":1}
I20260926 07:39:48.330708 15533 tablet_service.cc:1467] Processing CreateTablet for tablet 65a663298a644e35be294363195b526e (DEFAULT_TABLE table=test-table [id=d981356bbed14b2391f2f05357f27538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:48.331975 15533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a663298a644e35be294363195b526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:48.336292 15881 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Bootstrap starting.
I20260926 07:39:48.338353 15882 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Bootstrap starting.
I20260926 07:39:48.339895 15881 tablet_bootstrap.cc:654] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:48.341243 15881 log.cc:824] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:48.341744 15882 tablet_bootstrap.cc:654] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:48.342844 15882 log.cc:824] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:48.344278 15881 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: No bootstrap required, opened a new log
I20260926 07:39:48.344543 15881 ts_tablet_manager.cc:1397] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 07:39:48.345708 15882 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: No bootstrap required, opened a new log
I20260926 07:39:48.346029 15882 ts_tablet_manager.cc:1397] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 07:39:48.346052 15885 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Bootstrap starting.
I20260926 07:39:48.349128 15885 tablet_bootstrap.cc:654] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:48.350044 15885 log.cc:824] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:48.352995 15885 tablet_bootstrap.cc:492] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: No bootstrap required, opened a new log
I20260926 07:39:48.353315 15885 ts_tablet_manager.cc:1397] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260926 07:39:48.355870 15881 raft_consensus.cc:348] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.356338 15881 raft_consensus.cc:374] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:48.356443 15881 raft_consensus.cc:729] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f34046580e4e6f88e60b168a7ef5a2, State: Initialized, Role: FOLLOWER
I20260926 07:39:48.356949 15882 raft_consensus.cc:348] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.357069 15881 consensus_queue.cc:260] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.357442 15882 raft_consensus.cc:374] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:48.357583 15882 raft_consensus.cc:729] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf01ca302fe4b13b229c60736da97e5, State: Initialized, Role: FOLLOWER
I20260926 07:39:48.358371 15882 consensus_queue.cc:260] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.359649 15731 heartbeater.cc:503] Master 127.11.9.126:35865 was elected leader, sending a full tablet report...
I20260926 07:39:48.360662 15882 ts_tablet_manager.cc:1428] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 07:39:48.361150 15885 raft_consensus.cc:348] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.361619 15885 raft_consensus.cc:374] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:48.361744 15885 raft_consensus.cc:729] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeca62c2bf4a41c3aed5449f58b13072, State: Initialized, Role: FOLLOWER
I20260926 07:39:48.361925 15863 heartbeater.cc:503] Master 127.11.9.126:35865 was elected leader, sending a full tablet report...
I20260926 07:39:48.362417 15885 consensus_queue.cc:260] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.362593 15881 ts_tablet_manager.cc:1428] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260926 07:39:48.364034 15885 ts_tablet_manager.cc:1428] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
W20260926 07:39:48.486668 15864 tablet.cc:2369] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:39:48.492239 15732 tablet.cc:2369] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:39:48.508960 15599 tablet.cc:2369] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:48.581243 15888 raft_consensus.cc:482] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:48.581554 15888 raft_consensus.cc:504] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.583177 15888 leader_election.cc:288] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1f34046580e4e6f88e60b168a7ef5a2 (127.11.9.67:37569), eeca62c2bf4a41c3aed5449f58b13072 (127.11.9.65:36711)
I20260926 07:39:48.589622 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.583536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:46837 (local address 127.11.9.67:37569)
0926 07:39:48.583782 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.583788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.584107 (+ 319us) server_negotiation.cc:407] Connection header received
0926 07:39:48.584328 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.584335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.584448 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.584548 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:48.585319 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.586179 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.587937 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.588700 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.588847 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.588949 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.589228 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.589351 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:39:48.589716 15890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.583466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:37569 (local address 127.11.9.66:46837)
0926 07:39:48.583992 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.584009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.584181 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.584707 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.584712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.584747 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:48.585144 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.585154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.586304 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.586312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.587748 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.587807 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.588903 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.588909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.589071 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.589094 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.589223 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.589351 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":277,"threads_started":1}
I20260926 07:39:48.590184 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.584280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:34045 (local address 127.11.9.65:36711)
0926 07:39:48.584603 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.584609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.584918 (+ 309us) server_negotiation.cc:407] Connection header received
0926 07:39:48.585211 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.585216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.585272 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.585363 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:39:48.585908 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.586607 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.588454 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.589255 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.589396 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.589500 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.589872 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.589988 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 07:39:48.590714 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.584144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.11.9.66:34045)
0926 07:39:48.584798 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.584838 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:48.585035 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:48.585387 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:48.585391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:48.585414 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:39:48.585759 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.585766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.586886 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.586892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.588298 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:48.588309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.589479 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.589486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:48.589643 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.589668 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 07:39:48.590383 (+ 715us) client_negotiation.cc:240] Negotiation successful
0926 07:39:48.590502 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":405,"threads_started":1}
I20260926 07:39:48.590660 15818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a663298a644e35be294363195b526e" candidate_uuid: "0bf01ca302fe4b13b229c60736da97e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" is_pre_election: true
I20260926 07:39:48.591190 15818 raft_consensus.cc:2454] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf01ca302fe4b13b229c60736da97e5 in term 0.
I20260926 07:39:48.591753 15553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a663298a644e35be294363195b526e" candidate_uuid: "0bf01ca302fe4b13b229c60736da97e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeca62c2bf4a41c3aed5449f58b13072" is_pre_election: true
I20260926 07:39:48.591892 15619 leader_election.cc:302] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf01ca302fe4b13b229c60736da97e5, a1f34046580e4e6f88e60b168a7ef5a2; no voters:
I20260926 07:39:48.592255 15888 raft_consensus.cc:2781] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:48.592278 15553 raft_consensus.cc:2454] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf01ca302fe4b13b229c60736da97e5 in term 0.
I20260926 07:39:48.592403 15888 raft_consensus.cc:482] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:48.592520 15888 raft_consensus.cc:3037] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:48.595002 15888 raft_consensus.cc:504] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.595549 15888 leader_election.cc:288] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [CANDIDATE]: Term 1 election: Requested vote from peers a1f34046580e4e6f88e60b168a7ef5a2 (127.11.9.67:37569), eeca62c2bf4a41c3aed5449f58b13072 (127.11.9.65:36711)
I20260926 07:39:48.595896 15818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a663298a644e35be294363195b526e" candidate_uuid: "0bf01ca302fe4b13b229c60736da97e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1f34046580e4e6f88e60b168a7ef5a2"
I20260926 07:39:48.595973 15553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a663298a644e35be294363195b526e" candidate_uuid: "0bf01ca302fe4b13b229c60736da97e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeca62c2bf4a41c3aed5449f58b13072"
I20260926 07:39:48.596091 15818 raft_consensus.cc:3037] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:48.596189 15553 raft_consensus.cc:3037] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:48.598260 15818 raft_consensus.cc:2454] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf01ca302fe4b13b229c60736da97e5 in term 1.
I20260926 07:39:48.598407 15553 raft_consensus.cc:2454] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf01ca302fe4b13b229c60736da97e5 in term 1.
I20260926 07:39:48.598670 15619 leader_election.cc:302] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf01ca302fe4b13b229c60736da97e5, a1f34046580e4e6f88e60b168a7ef5a2; no voters:
I20260926 07:39:48.598917 15888 raft_consensus.cc:2781] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:48.599320 15888 raft_consensus.cc:686] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf01ca302fe4b13b229c60736da97e5, State: Running, Role: LEADER
I20260926 07:39:48.599762 15888 consensus_queue.cc:237] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } }
I20260926 07:39:48.604275 15409 catalog_manager.cc:5495] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bf01ca302fe4b13b229c60736da97e5 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "0bf01ca302fe4b13b229c60736da97e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bf01ca302fe4b13b229c60736da97e5" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 } health_report { overall_health: UNKNOWN } } }
I20260926 07:39:48.678247 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.672231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.11.9.66:33185)
0926 07:39:48.672411 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.672415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.672676 (+ 261us) server_negotiation.cc:407] Connection header received
0926 07:39:48.672852 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.672855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.672940 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.673072 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:39:48.673475 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.674158 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.676077 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.676501 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.676594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.677090 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:39:48.677682 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:39:48.677786 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.677922 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.677986 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:39:48.686312 15553 raft_consensus.cc:1261] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 1 FOLLOWER]: Refusing update from remote peer 0bf01ca302fe4b13b229c60736da97e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:48.686380 15818 raft_consensus.cc:1261] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 1 FOLLOWER]: Refusing update from remote peer 0bf01ca302fe4b13b229c60736da97e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:48.687184 15888 consensus_queue.cc:1035] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeca62c2bf4a41c3aed5449f58b13072" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:48.687495 15892 consensus_queue.cc:1035] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1f34046580e4e6f88e60b168a7ef5a2" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:48.698402 15895 mvcc.cc:205] Tried to move back new op lower bound from 7333512760049115136 to 7333512759708192768. Current Snapshot: MvccSnapshot[applied={T|T < 7333512760049115136}]
I20260926 07:39:48.710749 15408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:48.711057 15408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:48.714884 15408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce6e0e1d6c984c04a4a33ab0e740dbd7: Sending DeleteTablet for 3 replicas of tablet 65a663298a644e35be294363195b526e
I20260926 07:39:48.716032 15666 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a663298a644e35be294363195b526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:48 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:60990
I20260926 07:39:48.716073 15798 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a663298a644e35be294363195b526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:48 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:54264
I20260926 07:39:48.716459 15533 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a663298a644e35be294363195b526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:48 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:38922
I20260926 07:39:48.718374 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15471
I20260926 07:39:48.728037 15906 tablet_replica.cc:330] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: stopping tablet replica
I20260926 07:39:48.728547 15907 tablet_replica.cc:330] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: stopping tablet replica
I20260926 07:39:48.729511 15906 raft_consensus.cc:2229] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:48.729122 15907 raft_consensus.cc:2229] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:48.730296 15906 raft_consensus.cc:2258] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:48.730849 15907 raft_consensus.cc:2258] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:48.732878 15907 ts_tablet_manager.cc:1905] T 65a663298a644e35be294363195b526e P eeca62c2bf4a41c3aed5449f58b13072: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:48.734509 15906 ts_tablet_manager.cc:1905] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:48.742165 15905 tablet_replica.cc:330] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: stopping tablet replica
I20260926 07:39:48.742560 15905 raft_consensus.cc:2229] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:48.743068 15905 raft_consensus.cc:2258] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 07:39:48 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:48.745055 15905 ts_tablet_manager.cc:1905] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:48.751719 15905 ts_tablet_manager.cc:1918] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:48.751868 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.720769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.11.9.65:36711)
0926 07:39:48.720996 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.721012 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.724913 (+ 3901us) server_negotiation.cc:407] Connection header received
0926 07:39:48.725104 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.725114 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.725287 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.725412 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:48.729861 (+ 4449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.731380 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.734325 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.735201 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.735320 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.744865 (+ 9545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:48.744878 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:48.744894 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:48.744941 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:48.748598 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.749447 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.749455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.749459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.749615 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.750002 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.750007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.750011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.750400 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:48.750703 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.751261 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.751463 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:39:48.751911 15905 log.cc:1196] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/65a663298a644e35be294363195b526e
I20260926 07:39:48.752364 15905 ts_tablet_manager.cc:1939] T 65a663298a644e35be294363195b526e P a1f34046580e4e6f88e60b168a7ef5a2: Deleting consensus metadata
I20260926 07:39:48.753393 15906 ts_tablet_manager.cc:1918] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:48.753475 15593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:48.753546 15906 log.cc:1196] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/65a663298a644e35be294363195b526e
I20260926 07:39:48.754102 15906 ts_tablet_manager.cc:1939] T 65a663298a644e35be294363195b526e P 0bf01ca302fe4b13b229c60736da97e5: Deleting consensus metadata
I20260926 07:39:48.754195 15394 catalog_manager.cc:4841] TS a1f34046580e4e6f88e60b168a7ef5a2 (127.11.9.67:37569): tablet 65a663298a644e35be294363195b526e (table test-table [id=d981356bbed14b2391f2f05357f27538]) successfully deleted
I20260926 07:39:48.755477 15397 catalog_manager.cc:4841] TS 0bf01ca302fe4b13b229c60736da97e5 (127.11.9.66:33185): tablet 65a663298a644e35be294363195b526e (table test-table [id=d981356bbed14b2391f2f05357f27538]) successfully deleted
I20260926 07:39:48.878495 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15471
W20260926 07:39:48.895742 15394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:36711 (error 108)
I20260926 07:39:48.895998 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15604
I20260926 07:39:48.897928 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.897091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38940)
0926 07:39:48.897601 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.897725 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":295,"threads_started":1}
W20260926 07:39:48.898165 15394 catalog_manager.cc:4584] TS eeca62c2bf4a41c3aed5449f58b13072 (127.11.9.65:36711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65a663298a644e35be294363195b526e: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Sep 26 07:39:48 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:48.913045 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.896818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.11.9.66:33185)
0926 07:39:48.897000 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:48.897022 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:48.897065 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:39:48.897316 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:48.897325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:48.897431 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:48.897540 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:48.899679 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.900947 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.902437 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:48.902881 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:48.903045 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:48.907202 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:48.907215 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:48.907230 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:48.907268 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:48.910394 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.911164 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.911171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.911173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.911306 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:48.911676 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:48.911679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:48.911681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:48.911971 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:48.912222 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:48.912500 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 07:39:48.912690 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 07:39:48.913929 15726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:48.951886 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.951361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38950)
0926 07:39:48.951599 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.951713 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 07:39:48.952932 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:48.952529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38960)
0926 07:39:48.952688 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:48.952779 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 07:39:49.013509 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.012930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38964)
0926 07:39:49.013229 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:49.013338 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260926 07:39:49.014626 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.014266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38972)
0926 07:39:49.014396 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:49.014475 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 07:39:49.027133 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15604
I20260926 07:39:49.045034 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15736
Sep 26 07:39:49 dist-test-slave-x492 krb5kdc[15366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:49.059753 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.045875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.11.9.67:37569)
0926 07:39:49.046057 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:49.046064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:49.046141 (+ 77us) server_negotiation.cc:407] Connection header received
0926 07:39:49.046347 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:49.046353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:49.046428 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:49.046549 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:49.047789 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:49.048496 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:49.049933 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:49.050341 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:49.050471 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:49.054486 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:49.054515 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:49.054530 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:49.054570 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:49.056970 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:49.057748 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:49.057757 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:49.057761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:49.057905 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:49.058284 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:49.058289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:49.058293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:49.058637 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:49.058926 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:49.059226 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 07:39:49.059441 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:39:49.060468 15858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:49.095055 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.094606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38980)
0926 07:39:49.094813 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:49.094908 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 07:39:49.096060 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.095672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38982)
0926 07:39:49.095862 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:49.095928 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 07:39:49.147245 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15736
I20260926 07:39:49.163293 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15379
I20260926 07:39:49.164366 15440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:49.270057 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.269216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36711 (local address 127.0.0.1:38990)
0926 07:39:49.269793 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:49.269903 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260926 07:39:49.270115 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15379
2026-09-26T07:39:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4323 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T07:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:49Z Disabled control of system clock
I20260926 07:39:49.334396 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42785
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 07:39:49.645346 15918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:49.645615 15918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:49.645699 15918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:49.652487 15918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:49.652598 15918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:49.652642 15918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:49.652690 15918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:49.652726 15918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:49.661463 15918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:49.662653 15918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:49.664331 15918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:49.676101 15923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:49.676045 15926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:49.676040 15924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:49.676837 15918 server_base.cc:1034] running on GCE node
I20260926 07:39:49.677590 15918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:49.678805 15918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:49.680027 15918 hybrid_clock.cc:648] HybridClock initialized: now 1790408389679981 us; error 36 us; skew 500 ppm
I20260926 07:39:49.680357 15918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:49.682457 15918 webserver.cc:460] Webserver started at http://127.11.9.126:38905/ using document root <none> and password file <none>
I20260926 07:39:49.683104 15918 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:49.683228 15918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:49.683547 15918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:49.686311 15918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "45083b5e3af940f1aca32a8002514987"
format_stamp: "Formatted at 2026-09-26 07:39:49 on dist-test-slave-x492"
server_key: "1cf8b5fb79aae61b5fe89d3fb1bf4e61"
server_key_iv: "d8907d64c225b59064b5bbc23a699ff4"
server_key_version: "encryptionkey@0"
I20260926 07:39:49.687148 15918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "45083b5e3af940f1aca32a8002514987"
format_stamp: "Formatted at 2026-09-26 07:39:49 on dist-test-slave-x492"
server_key: "1cf8b5fb79aae61b5fe89d3fb1bf4e61"
server_key_iv: "d8907d64c225b59064b5bbc23a699ff4"
server_key_version: "encryptionkey@0"
I20260926 07:39:49.692181 15918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:39:49.695369 15932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:49.696640 15918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:49.696816 15918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "45083b5e3af940f1aca32a8002514987"
format_stamp: "Formatted at 2026-09-26 07:39:49 on dist-test-slave-x492"
server_key: "1cf8b5fb79aae61b5fe89d3fb1bf4e61"
server_key_iv: "d8907d64c225b59064b5bbc23a699ff4"
server_key_version: "encryptionkey@0"
I20260926 07:39:49.697026 15918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:49.702528 15918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:49.703227 15918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:49.703528 15918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:49.720753 15918 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42785
I20260926 07:39:49.720788 15983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42785 every 8 connection(s)
I20260926 07:39:49.722028 15918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:49.725458 15984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:49.730443 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 15918
I20260926 07:39:49.730628 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:49.731230 11301 external_mini_cluster.cc:1428] Setting key 36d29fd15380cc3175c2b7159b95644b
I20260926 07:39:49.736146 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: Bootstrap starting.
I20260926 07:39:49.739378 15984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:49.739764 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:49.733855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.11.9.126:42785)
0926 07:39:49.734769 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:49.734779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:49.734817 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:39:49.736397 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:49.736416 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:49.736778 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:49.737122 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:49.738186 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:49.738192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:49.738203 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:49.738215 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:49.738269 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:49.738283 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:49.738495 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:49.738627 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 07:39:49.738870 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":303,"threads_started":1}
I20260926 07:39:49.740409 15984 log.cc:824] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:49.743098 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: No bootstrap required, opened a new log
I20260926 07:39:49.750160 15984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } }
I20260926 07:39:49.750486 15984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:49.750598 15984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45083b5e3af940f1aca32a8002514987, State: Initialized, Role: FOLLOWER
I20260926 07:39:49.751196 15984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } }
I20260926 07:39:49.751420 15984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:49.751552 15984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:49.751721 15984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:49.753794 15984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } }
I20260926 07:39:49.754237 15984 leader_election.cc:302] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45083b5e3af940f1aca32a8002514987; no voters:
I20260926 07:39:49.754743 15984 leader_election.cc:288] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:49.754842 15989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:49.755573 15989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [term 1 LEADER]: Becoming Leader. State: Replica: 45083b5e3af940f1aca32a8002514987, State: Running, Role: LEADER
I20260926 07:39:49.756115 15989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } }
I20260926 07:39:49.756745 15984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:49.758733 15991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45083b5e3af940f1aca32a8002514987. Latest consensus state: current_term: 1 leader_uuid: "45083b5e3af940f1aca32a8002514987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } } }
I20260926 07:39:49.759068 15991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:49.760704 15996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:49.760794 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45083b5e3af940f1aca32a8002514987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45083b5e3af940f1aca32a8002514987" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42785 } } }
I20260926 07:39:49.761198 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:49.766533 15996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:49.774729 15996 catalog_manager.cc:1347] Generated new cluster ID: ca4dec5e29984a9faf908acefa4c1b54
I20260926 07:39:49.774838 15996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:49.806099 15996 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:49.806874 15996 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:49.812675 15996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: Generated new TSK 0
I20260926 07:39:49.813519 15996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:39:49.822927 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--builtin_ntp_servers=127.11.9.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 07:39:50.146624 16009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:50.147034 16009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:50.147181 16009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:50.154505 16009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:50.154596 16009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:50.154793 16009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:50.163895 16009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:50.165133 16009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:50.166896 16009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:50.179965 16014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:50.179935 16017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:50.180006 16015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:50.180877 16009 server_base.cc:1034] running on GCE node
I20260926 07:39:50.181483 16009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:50.182400 16009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:50.183626 16009 hybrid_clock.cc:648] HybridClock initialized: now 1790408390183586 us; error 49 us; skew 500 ppm
I20260926 07:39:50.183934 16009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:50.186123 16009 webserver.cc:460] Webserver started at http://127.11.9.65:34051/ using document root <none> and password file <none>
I20260926 07:39:50.186800 16009 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:50.186930 16009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:50.187265 16009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:50.190173 16009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "122468ba975b4abeadb68143ff18247e"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "275e5d841fb71a11ced6e4b14e30ee6f"
server_key_iv: "205a66287a346e468304484bbd747e35"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.190958 16009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "122468ba975b4abeadb68143ff18247e"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "275e5d841fb71a11ced6e4b14e30ee6f"
server_key_iv: "205a66287a346e468304484bbd747e35"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.195950 16009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:39:50.199426 16023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:50.200788 16009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:50.200948 16009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "122468ba975b4abeadb68143ff18247e"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "275e5d841fb71a11ced6e4b14e30ee6f"
server_key_iv: "205a66287a346e468304484bbd747e35"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.201162 16009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:50.206673 16009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:50.207418 16009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:50.207695 16009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:50.208837 16009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:50.210256 16009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:50.210389 16009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:50.210528 16009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:50.210616 16009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:50.243394 16009 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:34149
I20260926 07:39:50.243445 16135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:34149 every 8 connection(s)
I20260926 07:39:50.244750 16009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:50.252692 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16009
I20260926 07:39:50.252729 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:50.247325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:43315 (local address 127.11.9.126:42785)
0926 07:39:50.247740 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:50.247744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:50.248981 (+ 1237us) server_negotiation.cc:407] Connection header received
0926 07:39:50.250656 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:50.250660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:50.250708 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:50.250810 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:50.252064 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:50.252068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:50.252071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:50.252082 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:50.252115 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:50.252125 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:50.252218 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:50.252460 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 07:39:50.252505 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":226,"threads_started":1}
I20260926 07:39:50.252904 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:50.253782 11301 external_mini_cluster.cc:1428] Setting key 0d7477ae359d303be4fcce9b641ac445
I20260926 07:39:50.254117 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:50.247963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42785 (local address 127.11.9.65:43315)
0926 07:39:50.248829 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:50.248869 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:50.250370 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:50.251003 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:50.251011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:50.251097 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:50.251784 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:50.251787 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:50.251823 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:50.251916 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:50.252242 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:50.252252 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:50.252260 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 07:39:50.252575 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 07:39:50.252846 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":287,"threads_started":1}
I20260926 07:39:50.255611 16136 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42785
I20260926 07:39:50.255911 16136 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:50.256450 16136 heartbeater.cc:511] Master 127.11.9.126:42785 requested a full tablet report, sending...
I20260926 07:39:50.256495 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--builtin_ntp_servers=127.11.9.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 07:39:50.258728 15949 ts_manager.cc:194] Registered new tserver with Master: 122468ba975b4abeadb68143ff18247e (127.11.9.65:34149)
W20260926 07:39:50.569759 16140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:50.570067 16140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:50.570228 16140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:50.577271 16140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:50.577451 16140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:50.577622 16140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:50.586582 16140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:50.587864 16140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:50.589735 16140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:50.602697 16145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:50.602746 16146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:50.602962 16148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:50.603643 16140 server_base.cc:1034] running on GCE node
I20260926 07:39:50.604159 16140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:50.605015 16140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:50.606206 16140 hybrid_clock.cc:648] HybridClock initialized: now 1790408390606156 us; error 67 us; skew 500 ppm
I20260926 07:39:50.606510 16140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:50.608501 16140 webserver.cc:460] Webserver started at http://127.11.9.66:43559/ using document root <none> and password file <none>
I20260926 07:39:50.609150 16140 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:50.609313 16140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:50.609638 16140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:50.612304 16140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "8a22e5a00f7845c5a898be329e699fee"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "7c320e090206c9ebfaadf035645ad652"
server_key_iv: "003ab723ff58d0bb195897b52b332206"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.613289 16140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "8a22e5a00f7845c5a898be329e699fee"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "7c320e090206c9ebfaadf035645ad652"
server_key_iv: "003ab723ff58d0bb195897b52b332206"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.618577 16140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 07:39:50.622128 16154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:50.623616 16140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:39:50.623796 16140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "8a22e5a00f7845c5a898be329e699fee"
format_stamp: "Formatted at 2026-09-26 07:39:50 on dist-test-slave-x492"
server_key: "7c320e090206c9ebfaadf035645ad652"
server_key_iv: "003ab723ff58d0bb195897b52b332206"
server_key_version: "encryptionkey@0"
I20260926 07:39:50.624049 16140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:50.629669 16140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:50.630460 16140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:50.630730 16140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:50.631634 16140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:50.633147 16140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:50.633339 16140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 07:39:50.633509 16140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:50.633603 16140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:50.666492 16140 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:35231
I20260926 07:39:50.666510 16266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:35231 every 8 connection(s)
I20260926 07:39:50.667799 16140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:50.675105 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16140
I20260926 07:39:50.675287 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:50.675940 11301 external_mini_cluster.cc:1428] Setting key 56182423282ce3c1d087da1f4e70fc78
I20260926 07:39:50.676151 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:50.670356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:53039 (local address 127.11.9.126:42785)
0926 07:39:50.670568 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:50.670574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:50.672034 (+ 1460us) server_negotiation.cc:407] Connection header received
0926 07:39:50.673703 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:50.673707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:50.673771 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:50.673862 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:50.675387 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:50.675393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:50.675399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:50.675421 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:50.675460 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:50.675477 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:50.675619 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:50.675780 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 07:39:50.675841 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:39:50.676931 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:50.670937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42785 (local address 127.11.9.66:53039)
0926 07:39:50.671861 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:50.671909 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:50.673446 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:50.674213 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:50.674222 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:50.674307 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:50.675067 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:50.675070 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:50.675108 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:50.675188 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:50.675591 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:50.675598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:50.675604 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 07:39:50.675807 (+ 203us) client_negotiation.cc:240] Negotiation successful
0926 07:39:50.675984 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":246,"threads_started":1}
I20260926 07:39:50.678406 16267 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42785
I20260926 07:39:50.678685 16267 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:50.678934 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--builtin_ntp_servers=127.11.9.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 07:39:50.679263 16267 heartbeater.cc:511] Master 127.11.9.126:42785 requested a full tablet report, sending...
I20260926 07:39:50.680619 15949 ts_manager.cc:194] Registered new tserver with Master: 8a22e5a00f7845c5a898be329e699fee (127.11.9.66:35231)
W20260926 07:39:50.997712 16270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:50.998015 16270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:50.998172 16270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:51.005519 16270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:51.005637 16270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:51.005805 16270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:51.015091 16270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:51.016446 16270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:51.018344 16270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:51.031936 16275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:51.031963 16278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:51.031858 16276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:51.032680 16270 server_base.cc:1034] running on GCE node
I20260926 07:39:51.033384 16270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:51.034348 16270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:51.035609 16270 hybrid_clock.cc:648] HybridClock initialized: now 1790408391035538 us; error 61 us; skew 500 ppm
I20260926 07:39:51.035928 16270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:51.038244 16270 webserver.cc:460] Webserver started at http://127.11.9.67:44981/ using document root <none> and password file <none>
I20260926 07:39:51.038929 16270 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:51.039050 16270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:51.039381 16270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:51.042173 16270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "35a20ac906154e0ca9ba659b603bfad8"
format_stamp: "Formatted at 2026-09-26 07:39:51 on dist-test-slave-x492"
server_key: "71b116f2332995cadc6bcb74b864a26d"
server_key_iv: "bb69d4286a82cb1bd9932b000c9085ac"
server_key_version: "encryptionkey@0"
I20260926 07:39:51.042939 16270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "35a20ac906154e0ca9ba659b603bfad8"
format_stamp: "Formatted at 2026-09-26 07:39:51 on dist-test-slave-x492"
server_key: "71b116f2332995cadc6bcb74b864a26d"
server_key_iv: "bb69d4286a82cb1bd9932b000c9085ac"
server_key_version: "encryptionkey@0"
I20260926 07:39:51.047821 16270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:39:51.051457 16284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:51.052832 16270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:51.053015 16270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "35a20ac906154e0ca9ba659b603bfad8"
format_stamp: "Formatted at 2026-09-26 07:39:51 on dist-test-slave-x492"
server_key: "71b116f2332995cadc6bcb74b864a26d"
server_key_iv: "bb69d4286a82cb1bd9932b000c9085ac"
server_key_version: "encryptionkey@0"
I20260926 07:39:51.053243 16270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:51.058734 16270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:51.059482 16270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:51.059744 16270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:51.060657 16270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:51.062060 16270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:51.062176 16270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:51.062305 16270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:51.062407 16270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:51.094707 16270 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:33869
I20260926 07:39:51.094758 16396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:33869 every 8 connection(s)
I20260926 07:39:51.096043 16270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:51.096997 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16270
I20260926 07:39:51.097205 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:51.097759 11301 external_mini_cluster.cc:1428] Setting key 5b9b3cd81903bfe0f641e15e924e8847
I20260926 07:39:51.104909 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.098949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:58677 (local address 127.11.9.126:42785)
0926 07:39:51.099154 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.099159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.101096 (+ 1937us) server_negotiation.cc:407] Connection header received
0926 07:39:51.102815 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.102820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.102882 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.102999 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.104206 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.104211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.104216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.104235 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.104274 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.104290 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.104391 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.104643 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.104693 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:39:51.105978 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.099754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42785 (local address 127.11.9.67:58677)
0926 07:39:51.100937 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.100984 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.102544 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.103225 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.103233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.103321 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.103920 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.103922 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.103957 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.104036 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.104367 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.104376 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.104384 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.104697 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.104881 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":366,"threads_started":1}
I20260926 07:39:51.107245 16397 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42785
I20260926 07:39:51.107489 16397 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:51.107972 16397 heartbeater.cc:511] Master 127.11.9.126:42785 requested a full tablet report, sending...
I20260926 07:39:51.109241 15949 ts_manager.cc:194] Registered new tserver with Master: 35a20ac906154e0ca9ba659b603bfad8 (127.11.9.67:33869)
I20260926 07:39:51.113024 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:51.119374 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.117204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.11.9.126:42785)
0926 07:39:51.117343 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.117347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.117866 (+ 519us) server_negotiation.cc:407] Connection header received
0926 07:39:51.117983 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.117986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.118032 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.118109 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.118953 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.118956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.118959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.118970 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.119004 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.119014 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.119107 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.119159 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.119201 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 07:39:51.123157 15949 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:39:51.129994 15949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:39:51.147068 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.141835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.11.9.65:34149)
0926 07:39:51.142883 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.142895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.143908 (+ 1013us) server_negotiation.cc:407] Connection header received
0926 07:39:51.144165 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.144173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.144386 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.144514 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.145824 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.145836 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.145853 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.145877 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.145953 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.145978 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.146140 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.146657 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.146760 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":1414,"threads_started":1}
I20260926 07:39:51.147081 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.141244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:35231 (local address 127.0.0.1:58488)
0926 07:39:51.142073 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.142124 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.142286 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.143785 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.143792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.143906 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.144774 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.144777 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.144806 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.144887 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.146399 (+ 1512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.146403 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.146408 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.146530 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.146616 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":726,"threads_started":1}
I20260926 07:39:51.147758 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.141459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36814)
0926 07:39:51.143792 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.143811 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.143995 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.144923 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.144930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.144957 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.145597 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.145602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.145620 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.145661 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.146198 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.146212 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.146219 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.146405 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.147302 (+ 897us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2195,"mutex_wait_us":679,"thread_start_us":2154,"threads_started":1}
I20260926 07:39:51.148067 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.141540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.11.9.66:35231)
0926 07:39:51.143287 (+ 1747us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.143295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.143321 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:39:51.143389 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.143396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.143607 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.143766 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.145073 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.145083 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.145100 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.145121 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.145209 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.145234 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.147521 (+ 2287us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.147608 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.147699 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1594,"thread_start_us":1488,"threads_started":1}
I20260926 07:39:51.149415 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.144488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.11.9.67:33869)
0926 07:39:51.146005 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.146012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.146537 (+ 525us) server_negotiation.cc:407] Connection header received
0926 07:39:51.146656 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.146664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.146850 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.146997 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.148578 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.148585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.148599 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.148616 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.148668 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.148688 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.148863 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.148927 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.149027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1362,"thread_start_us":938,"threads_started":1}
I20260926 07:39:51.149618 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.143955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:33869 (local address 127.0.0.1:43242)
0926 07:39:51.146405 (+ 2450us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.146420 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.146515 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.147014 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.147019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.147049 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.148350 (+ 1301us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.148353 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.148371 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.148418 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.148803 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.148808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.148812 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.148943 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.148992 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2322,"thread_start_us":2168,"threads_started":1}
I20260926 07:39:51.151530 16202 tablet_service.cc:1467] Processing CreateTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed (DEFAULT_TABLE table=test-table [id=939da37dc32b4d7ab1c0f0182a03bb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:51.151660 16071 tablet_service.cc:1467] Processing CreateTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed (DEFAULT_TABLE table=test-table [id=939da37dc32b4d7ab1c0f0182a03bb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:51.152724 16202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f8f41b6ff9448ea1999eb5201569ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:51.152930 16071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f8f41b6ff9448ea1999eb5201569ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:51.153196 16332 tablet_service.cc:1467] Processing CreateTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed (DEFAULT_TABLE table=test-table [id=939da37dc32b4d7ab1c0f0182a03bb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:39:51.154433 16332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8f8f41b6ff9448ea1999eb5201569ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:51.164615 16415 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Bootstrap starting.
I20260926 07:39:51.167469 16416 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Bootstrap starting.
I20260926 07:39:51.169008 16415 tablet_bootstrap.cc:654] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:51.169116 16417 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Bootstrap starting.
I20260926 07:39:51.170470 16415 log.cc:824] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:51.171119 16416 tablet_bootstrap.cc:654] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:51.172122 16417 tablet_bootstrap.cc:654] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:51.172206 16416 log.cc:824] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:51.173246 16417 log.cc:824] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:51.173722 16415 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: No bootstrap required, opened a new log
I20260926 07:39:51.173985 16415 ts_tablet_manager.cc:1397] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 07:39:51.176028 16416 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: No bootstrap required, opened a new log
I20260926 07:39:51.176281 16416 ts_tablet_manager.cc:1397] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:39:51.179118 16417 tablet_bootstrap.cc:492] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: No bootstrap required, opened a new log
I20260926 07:39:51.179364 16417 ts_tablet_manager.cc:1397] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260926 07:39:51.184664 16415 raft_consensus.cc:348] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.185181 16415 raft_consensus.cc:374] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:51.185292 16415 raft_consensus.cc:729] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a22e5a00f7845c5a898be329e699fee, State: Initialized, Role: FOLLOWER
I20260926 07:39:51.185900 16415 consensus_queue.cc:260] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.186955 16267 heartbeater.cc:503] Master 127.11.9.126:42785 was elected leader, sending a full tablet report...
I20260926 07:39:51.186825 16416 raft_consensus.cc:348] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.187228 16416 raft_consensus.cc:374] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:51.187368 16416 raft_consensus.cc:729] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122468ba975b4abeadb68143ff18247e, State: Initialized, Role: FOLLOWER
I20260926 07:39:51.187702 16415 ts_tablet_manager.cc:1428] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260926 07:39:51.187986 16416 consensus_queue.cc:260] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.186688 16417 raft_consensus.cc:348] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.188467 16417 raft_consensus.cc:374] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:51.188581 16417 raft_consensus.cc:729] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a20ac906154e0ca9ba659b603bfad8, State: Initialized, Role: FOLLOWER
I20260926 07:39:51.189404 16417 consensus_queue.cc:260] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.189718 16423 raft_consensus.cc:482] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:51.189944 16423 raft_consensus.cc:504] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.190608 16136 heartbeater.cc:503] Master 127.11.9.126:42785 was elected leader, sending a full tablet report...
I20260926 07:39:51.191319 16416 ts_tablet_manager.cc:1428] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260926 07:39:51.195261 16423 leader_election.cc:288] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a22e5a00f7845c5a898be329e699fee (127.11.9.66:35231), 122468ba975b4abeadb68143ff18247e (127.11.9.65:34149)
I20260926 07:39:51.195920 16397 heartbeater.cc:503] Master 127.11.9.126:42785 was elected leader, sending a full tablet report...
I20260926 07:39:51.196414 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.192097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:35231 (local address 127.11.9.67:48031)
0926 07:39:51.192528 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.192551 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.192648 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.194609 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.194617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.194651 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.195442 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.195448 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.195470 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.195527 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.195984 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.195994 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.196000 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.196103 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.196157 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304}
I20260926 07:39:51.197285 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.193341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:48031 (local address 127.11.9.66:35231)
0926 07:39:51.193849 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.193857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.193881 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:39:51.193968 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.193976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.194058 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.194172 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.195730 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.195739 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.195744 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.195762 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.195797 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.195812 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.195928 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.196964 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.197019 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260926 07:39:51.198377 16222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f8f41b6ff9448ea1999eb5201569ed" candidate_uuid: "35a20ac906154e0ca9ba659b603bfad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a22e5a00f7845c5a898be329e699fee" is_pre_election: true
I20260926 07:39:51.198705 16417 ts_tablet_manager.cc:1428] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Time spent starting tablet: real 0.019s user 0.010s sys 0.007s
I20260926 07:39:51.198851 16222 raft_consensus.cc:2454] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a20ac906154e0ca9ba659b603bfad8 in term 0.
I20260926 07:39:51.199281 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.196736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:43937 (local address 127.11.9.65:34149)
0926 07:39:51.196990 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.196995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.197010 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:39:51.197054 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.197078 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.197127 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.197238 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.198622 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.198627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.198631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.198648 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.198685 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.198701 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.198803 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.198983 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.199034 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260926 07:39:51.199337 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.195797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.11.9.67:43937)
0926 07:39:51.196633 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.196652 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:51.196723 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:51.197570 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:51.197575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:51.197597 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:51.198401 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:39:51.198404 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:51.198420 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:39:51.198465 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:51.198851 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:51.198856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:51.198861 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:39:51.198984 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 07:39:51.199030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"threads_started":1}
I20260926 07:39:51.199544 16288 leader_election.cc:302] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a20ac906154e0ca9ba659b603bfad8, 8a22e5a00f7845c5a898be329e699fee; no voters:
I20260926 07:39:51.199923 16423 raft_consensus.cc:2781] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:39:51.200093 16423 raft_consensus.cc:482] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:39:51.200208 16423 raft_consensus.cc:3037] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:51.201442 16091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f8f41b6ff9448ea1999eb5201569ed" candidate_uuid: "35a20ac906154e0ca9ba659b603bfad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "122468ba975b4abeadb68143ff18247e" is_pre_election: true
I20260926 07:39:51.201866 16091 raft_consensus.cc:2454] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a20ac906154e0ca9ba659b603bfad8 in term 0.
I20260926 07:39:51.202924 16423 raft_consensus.cc:504] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.203456 16423 leader_election.cc:288] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a22e5a00f7845c5a898be329e699fee (127.11.9.66:35231), 122468ba975b4abeadb68143ff18247e (127.11.9.65:34149)
I20260926 07:39:51.203881 16091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f8f41b6ff9448ea1999eb5201569ed" candidate_uuid: "35a20ac906154e0ca9ba659b603bfad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "122468ba975b4abeadb68143ff18247e"
I20260926 07:39:51.203953 16222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8f8f41b6ff9448ea1999eb5201569ed" candidate_uuid: "35a20ac906154e0ca9ba659b603bfad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a22e5a00f7845c5a898be329e699fee"
I20260926 07:39:51.204067 16091 raft_consensus.cc:3037] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:51.204149 16222 raft_consensus.cc:3037] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:51.206457 16091 raft_consensus.cc:2454] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a20ac906154e0ca9ba659b603bfad8 in term 1.
I20260926 07:39:51.206733 16222 raft_consensus.cc:2454] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a20ac906154e0ca9ba659b603bfad8 in term 1.
I20260926 07:39:51.206863 16286 leader_election.cc:302] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 122468ba975b4abeadb68143ff18247e, 35a20ac906154e0ca9ba659b603bfad8; no voters:
I20260926 07:39:51.207196 16423 raft_consensus.cc:2781] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:51.207340 16423 raft_consensus.cc:686] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 1 LEADER]: Becoming Leader. State: Replica: 35a20ac906154e0ca9ba659b603bfad8, State: Running, Role: LEADER
I20260926 07:39:51.207772 16423 consensus_queue.cc:237] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } }
I20260926 07:39:51.212165 15949 catalog_manager.cc:5495] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35a20ac906154e0ca9ba659b603bfad8 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "35a20ac906154e0ca9ba659b603bfad8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35a20ac906154e0ca9ba659b603bfad8" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33869 } health_report { overall_health: HEALTHY } } }
I20260926 07:39:51.233862 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.231717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.11.9.67:33869)
0926 07:39:51.231912 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.231919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.231982 (+ 63us) server_negotiation.cc:407] Connection header received
0926 07:39:51.232153 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.232157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.232230 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.232315 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.233266 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.233269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.233272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.233285 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.233311 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.233321 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.233412 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.233540 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.233582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 07:39:51.241712 16091 raft_consensus.cc:1261] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 1 FOLLOWER]: Refusing update from remote peer 35a20ac906154e0ca9ba659b603bfad8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:51.241715 16222 raft_consensus.cc:1261] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 1 FOLLOWER]: Refusing update from remote peer 35a20ac906154e0ca9ba659b603bfad8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:39:51.242666 16424 consensus_queue.cc:1035] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "122468ba975b4abeadb68143ff18247e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:39:51.243057 16423 consensus_queue.cc:1035] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a22e5a00f7845c5a898be329e699fee" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:39:51.247078 16137 tablet.cc:2369] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:39:51.262506 16428 mvcc.cc:205] Tried to move back new op lower bound from 7333512770516389888 to 7333512770391158784. Current Snapshot: MvccSnapshot[applied={T|T < 7333512770516389888}]
I20260926 07:39:51.263018 16430 mvcc.cc:205] Tried to move back new op lower bound from 7333512770516389888 to 7333512770391158784. Current Snapshot: MvccSnapshot[applied={T|T < 7333512770516389888}]
I20260926 07:39:51.276134 15949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:51.276394 15949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:39:51.279577 15949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45083b5e3af940f1aca32a8002514987: Sending DeleteTablet for 3 replicas of tablet e8f8f41b6ff9448ea1999eb5201569ed
I20260926 07:39:51.280627 16202 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:51 UTC) from {username='slave'} at 127.0.0.1:58488
I20260926 07:39:51.280756 16071 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:51 UTC) from {username='slave'} at 127.0.0.1:36814
I20260926 07:39:51.281587 16332 tablet_service.cc:1514] Processing DeleteTablet for tablet e8f8f41b6ff9448ea1999eb5201569ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:39:51 UTC) from {username='slave'} at 127.0.0.1:43242
I20260926 07:39:51.282200 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16009
I20260926 07:39:51.295986 16437 tablet_replica.cc:330] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: stopping tablet replica
I20260926 07:39:51.296489 16437 raft_consensus.cc:2229] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:51.296998 16437 raft_consensus.cc:2258] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:51.297472 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.283801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.11.9.65:34149)
0926 07:39:51.284039 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.284047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.294358 (+ 10311us) server_negotiation.cc:407] Connection header received
0926 07:39:51.294876 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.294885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.295033 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.295124 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.296478 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.296486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.296490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.296510 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.296548 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.296565 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.296663 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.297016 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.297082 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:39:51.298681 16131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:51.299098 16437 ts_tablet_manager.cc:1905] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:51.301103 16439 tablet_replica.cc:330] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: stopping tablet replica
I20260926 07:39:51.301771 16439 raft_consensus.cc:2229] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:39:51.302670 16439 raft_consensus.cc:2258] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:39:51.305419 16439 ts_tablet_manager.cc:1905] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:39:51.307454 16437 ts_tablet_manager.cc:1918] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:51.307615 16437 log.cc:1196] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/e8f8f41b6ff9448ea1999eb5201569ed
I20260926 07:39:51.308113 16437 ts_tablet_manager.cc:1939] T e8f8f41b6ff9448ea1999eb5201569ed P 8a22e5a00f7845c5a898be329e699fee: Deleting consensus metadata
I20260926 07:39:51.309950 15936 catalog_manager.cc:4841] TS 8a22e5a00f7845c5a898be329e699fee (127.11.9.66:35231): tablet e8f8f41b6ff9448ea1999eb5201569ed (table test-table [id=939da37dc32b4d7ab1c0f0182a03bb0b]) successfully deleted
I20260926 07:39:51.315480 16439 ts_tablet_manager.cc:1918] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:39:51.315834 16439 log.cc:1196] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/e8f8f41b6ff9448ea1999eb5201569ed
I20260926 07:39:51.316489 16439 ts_tablet_manager.cc:1939] T e8f8f41b6ff9448ea1999eb5201569ed P 35a20ac906154e0ca9ba659b603bfad8: Deleting consensus metadata
I20260926 07:39:51.317997 15934 catalog_manager.cc:4841] TS 35a20ac906154e0ca9ba659b603bfad8 (127.11.9.67:33869): tablet e8f8f41b6ff9448ea1999eb5201569ed (table test-table [id=939da37dc32b4d7ab1c0f0182a03bb0b]) successfully deleted
I20260926 07:39:51.423887 16438 tablet_replica.cc:330] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e: stopping tablet replica
I20260926 07:39:51.424468 16438 raft_consensus.cc:2229] T e8f8f41b6ff9448ea1999eb5201569ed P 122468ba975b4abeadb68143ff18247e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:39:51.424626 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16009
I20260926 07:39:51.448864 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16140
W20260926 07:39:51.448864 15934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:34149 (error 108)
I20260926 07:39:51.450917 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.450413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36830)
0926 07:39:51.450610 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.450742 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 07:39:51.451210 15934 catalog_manager.cc:4584] TS 122468ba975b4abeadb68143ff18247e (127.11.9.65:34149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8f8f41b6ff9448ea1999eb5201569ed: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
I20260926 07:39:51.452524 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.450132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.11.9.66:35231)
0926 07:39:51.450306 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.450314 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.450334 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:39:51.450404 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.450409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.450518 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.450666 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.451858 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.451869 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.451873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.451887 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.451914 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.451925 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.452077 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.452175 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.452244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 07:39:51.453142 16262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:51.504920 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.504417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36842)
0926 07:39:51.504629 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.504712 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 07:39:51.506036 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.505700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36856)
0926 07:39:51.505841 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.505928 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 07:39:51.566419 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.565934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36868)
0926 07:39:51.566159 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.566268 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 07:39:51.567499 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.567168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36878)
0926 07:39:51.567315 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.567390 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 07:39:51.600088 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16140
I20260926 07:39:51.616516 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16270
I20260926 07:39:51.619740 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.617342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.11.9.67:33869)
0926 07:39:51.617502 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:51.617508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:51.617581 (+ 73us) server_negotiation.cc:407] Connection header received
0926 07:39:51.617874 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:51.617878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:51.617937 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:51.618133 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:51.619032 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:51.619037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:51.619043 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:51.619063 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:51.619110 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:51.619121 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:51.619269 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:51.619334 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 07:39:51.619392 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:39:51.620466 16392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:51.648430 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.647854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36886)
0926 07:39:51.648131 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.648261 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 07:39:51.649518 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.649057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36890)
0926 07:39:51.649303 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.649383 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260926 07:39:51.737244 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16270
I20260926 07:39:51.756628 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 15918
I20260926 07:39:51.757845 15979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:51.860154 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:51.859590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:34149 (local address 127.0.0.1:36894)
0926 07:39:51.859848 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:51.859953 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260926 07:39:51.860366 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 15918
2026-09-26T07:39:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2588 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 07:39:51.901124 11301 mini_oidc.cc:152] Starting JWKS server
I20260926 07:39:51.903828 11301 webserver.cc:460] Webserver started at https://127.0.0.1:40569/ using document root <none> and password file <none>
I20260926 07:39:51.904143 11301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 07:39:51.904749 11301 webserver.cc:460] Webserver started at http://127.0.0.1:41923/ using document root <none> and password file <none>
2026-09-26T07:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:51Z Disabled control of system clock
I20260926 07:39:51.931036 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38729
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:38729
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40569/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/testchainca.cert with env {}
W20260926 07:39:52.250233 16451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:52.250581 16451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:52.250736 16451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:52.257753 16451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:52.257890 16451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:52.257984 16451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:52.258085 16451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:52.258172 16451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 07:39:52.258260 16451 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 07:39:52.258337 16451 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40569/jwks.json
I20260926 07:39:52.267784 16451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:38729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40569/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:52.269083 16451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:52.270375 16451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:52.281728 16457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:52.281781 16459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:52.281827 16456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:52.282691 16451 server_base.cc:1034] running on GCE node
I20260926 07:39:52.283427 16451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:52.284775 16451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:52.285997 16451 hybrid_clock.cc:648] HybridClock initialized: now 1790408392285954 us; error 61 us; skew 500 ppm
I20260926 07:39:52.286336 16451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:52.288559 16451 webserver.cc:460] Webserver started at http://127.11.9.126:35039/ using document root <none> and password file <none>
I20260926 07:39:52.289297 16451 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:52.289438 16451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:52.289744 16451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:52.292524 16451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "9837f88b71644db691e63b74dd0c6ffc"
format_stamp: "Formatted at 2026-09-26 07:39:52 on dist-test-slave-x492"
server_key: "db1212511d38c0fd87086587036f555a"
server_key_iv: "76478b0ddd832436171925030fbc1769"
server_key_version: "encryptionkey@0"
I20260926 07:39:52.293437 16451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "9837f88b71644db691e63b74dd0c6ffc"
format_stamp: "Formatted at 2026-09-26 07:39:52 on dist-test-slave-x492"
server_key: "db1212511d38c0fd87086587036f555a"
server_key_iv: "76478b0ddd832436171925030fbc1769"
server_key_version: "encryptionkey@0"
I20260926 07:39:52.298448 16451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:52.301889 16465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:52.303171 16451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:52.303359 16451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "9837f88b71644db691e63b74dd0c6ffc"
format_stamp: "Formatted at 2026-09-26 07:39:52 on dist-test-slave-x492"
server_key: "db1212511d38c0fd87086587036f555a"
server_key_iv: "76478b0ddd832436171925030fbc1769"
server_key_version: "encryptionkey@0"
I20260926 07:39:52.303592 16451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:52.319960 16451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:52.320782 16451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:52.321063 16451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:52.338980 16451 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:38729
I20260926 07:39:52.339000 16516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:38729 every 8 connection(s)
I20260926 07:39:52.340277 16451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:52.343961 16517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:52.348299 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16451
I20260926 07:39:52.348512 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:52.349099 11301 external_mini_cluster.cc:1428] Setting key f138387b3712ead7ad224fad29457f70
I20260926 07:39:52.354758 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: Bootstrap starting.
I20260926 07:39:52.357825 16517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:52.359136 16517 log.cc:824] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:52.362468 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: No bootstrap required, opened a new log
I20260926 07:39:52.363240 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:52.351114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.11.9.126:38729)
0926 07:39:52.352127 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:52.352138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:52.352181 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:39:52.353973 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:52.353995 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:52.354423 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:52.354804 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:52.355345 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.356612 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.358071 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.359175 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.359364 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:52.360547 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:52.360558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:52.360576 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:52.361251 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:52.361352 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:52.361378 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:52.361491 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:52.361687 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 07:39:52.361994 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":344,"threads_started":1}
I20260926 07:39:52.366962 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:39:52.370862 16517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } }
I20260926 07:39:52.371203 16517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:52.371327 16517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9837f88b71644db691e63b74dd0c6ffc, State: Initialized, Role: FOLLOWER
I20260926 07:39:52.371917 16517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } }
I20260926 07:39:52.372138 16517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:52.372282 16517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:52.372459 16517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:52.374754 16517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } }
I20260926 07:39:52.375239 16517 leader_election.cc:302] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9837f88b71644db691e63b74dd0c6ffc; no voters:
I20260926 07:39:52.375777 16517 leader_election.cc:288] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:52.375900 16522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:52.376657 16522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [term 1 LEADER]: Becoming Leader. State: Replica: 9837f88b71644db691e63b74dd0c6ffc, State: Running, Role: LEADER
I20260926 07:39:52.377220 16522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } }
I20260926 07:39:52.377902 16517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:52.380038 16523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9837f88b71644db691e63b74dd0c6ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } } }
I20260926 07:39:52.380452 16523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:52.384796 16524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9837f88b71644db691e63b74dd0c6ffc. Latest consensus state: current_term: 1 leader_uuid: "9837f88b71644db691e63b74dd0c6ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9837f88b71644db691e63b74dd0c6ffc" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38729 } } }
I20260926 07:39:52.385339 16524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc [sys.catalog]: This master's current role is: LEADER
W20260926 07:39:52.388639 16538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:39:52.388746 16538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:39:52.388939 16531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:52.390574 16531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:52.398785 16531 catalog_manager.cc:1347] Generated new cluster ID: bb4850c7bb8d4da2a499e592e100ae8f
I20260926 07:39:52.398931 16531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:52.415093 16531 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:52.416926 16531 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:52.437479 16531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9837f88b71644db691e63b74dd0c6ffc: Generated new TSK 0
I20260926 07:39:52.438414 16531 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 07:39:52.474076 16547 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 07:39:52.474344 16547 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 07:39:52.475286 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:52.452024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.11.9.126:38729)
0926 07:39:52.452274 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:52.452281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:52.452719 (+ 438us) server_negotiation.cc:407] Connection header received
0926 07:39:52.453059 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:52.453062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:52.453128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:52.453308 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0926 07:39:52.453955 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.454663 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.456750 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.457303 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.457430 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:52.458138 (+ 708us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 07:39:52.474747 (+ 16609us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 07:39:52.475038 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59816: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 07:39:52.475442 16520 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59816: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 07:39:52.477234 16546 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:52.451899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38729 (local address 127.0.0.1:59816)
0926 07:39:52.452600 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:52.452625 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:52.452814 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:52.453370 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:52.453375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:52.453389 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0926 07:39:52.453809 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:52.453822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.454802 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.454808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:52.456605 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:52.456620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:52.457543 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:52.457550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:52.457747 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:52.457824 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 07:39:52.475043 (+ 17219us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 07:39:52.475429 (+ 386us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.9.126:38729: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":359,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.9.126:38729: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 07:39:52.479041 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16451
I20260926 07:39:52.480314 16512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:52.573645 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16451
2026-09-26T07:39:52Z chronyd exiting
I20260926 07:39:52.624131 11301 test_util.cc:175] -----------------------------------------------
I20260926 07:39:52.624272 11301 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790408351541924-11301-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (727 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 07:39:52.628029 11301 mini_oidc.cc:152] Starting JWKS server
I20260926 07:39:52.629880 11301 webserver.cc:460] Webserver started at https://127.0.0.1:36415/ using document root <none> and password file <none>
I20260926 07:39:52.630107 11301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 07:39:52.630751 11301 webserver.cc:460] Webserver started at http://127.0.0.1:37591/ using document root <none> and password file <none>
2026-09-26T07:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:52Z Disabled control of system clock
I20260926 07:39:52.656591 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36395
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:40793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:36395
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36415/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/testchainca.cert with env {}
W20260926 07:39:52.970422 16554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:52.970741 16554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:52.970861 16554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:52.978147 16554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:52.978276 16554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:52.978360 16554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:52.978595 16554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:52.978828 16554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 07:39:52.978948 16554 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 07:39:52.979008 16554 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36415/jwks.json
I20260926 07:39:52.989640 16554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:36395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36415/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:52.990999 16554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:52.992339 16554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:53.004513 16559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:53.004498 16560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:53.004523 16562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:53.005416 16554 server_base.cc:1034] running on GCE node
I20260926 07:39:53.006213 16554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:53.007573 16554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:53.008797 16554 hybrid_clock.cc:648] HybridClock initialized: now 1790408393008745 us; error 46 us; skew 500 ppm
I20260926 07:39:53.009151 16554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:53.011555 16554 webserver.cc:460] Webserver started at http://127.11.9.126:41221/ using document root <none> and password file <none>
I20260926 07:39:53.012315 16554 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:53.012446 16554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:53.012764 16554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:53.015636 16554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "72f3757bb39840aea0c130a54bad8979"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "96fa17d3b9773e121cf820ad275868b5"
server_key_iv: "1c60b23ad5b6b2fcbf6ada60738b5a04"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.016412 16554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "72f3757bb39840aea0c130a54bad8979"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "96fa17d3b9773e121cf820ad275868b5"
server_key_iv: "1c60b23ad5b6b2fcbf6ada60738b5a04"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.021564 16554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 07:39:53.024817 16568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:53.026145 16554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:39:53.026326 16554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "72f3757bb39840aea0c130a54bad8979"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "96fa17d3b9773e121cf820ad275868b5"
server_key_iv: "1c60b23ad5b6b2fcbf6ada60738b5a04"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.026556 16554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:53.060776 16554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:53.061657 16554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:53.061949 16554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:53.080163 16554 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:36395
I20260926 07:39:53.080185 16619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:36395 every 8 connection(s)
I20260926 07:39:53.081537 16554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:53.083777 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16554
I20260926 07:39:53.083971 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:53.084753 11301 external_mini_cluster.cc:1428] Setting key bcd03df9935d143836d20a870d72429f
I20260926 07:39:53.085685 16620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:53.097630 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:53.086719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.11.9.126:36395)
0926 07:39:53.087644 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:53.087655 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:53.087699 (+ 44us) server_negotiation.cc:407] Connection header received
0926 07:39:53.089227 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:53.089261 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:53.089721 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:53.090110 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:53.090649 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.091941 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.093264 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.093887 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.094080 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.095308 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:53.095316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:53.095332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:53.095893 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:53.095948 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:53.095967 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:53.096091 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:53.096313 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 07:39:53.096596 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":344,"threads_started":1}
I20260926 07:39:53.097630 16620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979: Bootstrap starting.
I20260926 07:39:53.101120 16620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:53.101264 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:39:53.102124 16620 log.cc:824] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:53.104835 16620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979: No bootstrap required, opened a new log
I20260926 07:39:53.111948 16620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } }
I20260926 07:39:53.112294 16620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:53.112402 16620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f3757bb39840aea0c130a54bad8979, State: Initialized, Role: FOLLOWER
I20260926 07:39:53.112926 16620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } }
I20260926 07:39:53.113153 16620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:53.113327 16620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:53.113505 16620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:53.116335 16620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } }
I20260926 07:39:53.116827 16620 leader_election.cc:302] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72f3757bb39840aea0c130a54bad8979; no voters:
I20260926 07:39:53.117456 16620 leader_election.cc:288] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:53.117581 16625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:53.118294 16625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [term 1 LEADER]: Becoming Leader. State: Replica: 72f3757bb39840aea0c130a54bad8979, State: Running, Role: LEADER
I20260926 07:39:53.118908 16625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } }
I20260926 07:39:53.119619 16620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:53.121964 16627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72f3757bb39840aea0c130a54bad8979. Latest consensus state: current_term: 1 leader_uuid: "72f3757bb39840aea0c130a54bad8979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } } }
I20260926 07:39:53.122386 16627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:53.123226 16626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72f3757bb39840aea0c130a54bad8979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f3757bb39840aea0c130a54bad8979" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36395 } } }
I20260926 07:39:53.123581 16626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:53.123916 16634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:53.129204 16634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:53.137511 16634 catalog_manager.cc:1347] Generated new cluster ID: 2c32052ded5d42f09b001cb337e93be4
I20260926 07:39:53.137621 16634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:53.162144 16634 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:53.164121 16634 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:53.172401 16634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72f3757bb39840aea0c130a54bad8979: Generated new TSK 0
I20260926 07:39:53.173431 16634 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 07:39:53.208524 16650 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 07:39:53.208652 16650 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 07:39:53.209898 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:53.189496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.11.9.126:36395)
0926 07:39:53.189708 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:53.189714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:53.190110 (+ 396us) server_negotiation.cc:407] Connection header received
0926 07:39:53.190288 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:53.190293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:53.190379 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:53.190546 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0926 07:39:53.190939 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.192089 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.193547 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.194266 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.194397 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.194940 (+ 543us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 07:39:53.209240 (+ 14300us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 07:39:53.209574 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":79}
W20260926 07:39:53.209923 16649 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:53.189358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36395 (local address 127.0.0.1:38054)
0926 07:39:53.189992 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:53.190007 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:53.190131 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:53.190551 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:53.190554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:53.190562 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 07:39:53.190785 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:53.190792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.192240 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.192244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:53.193409 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:53.193420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.194516 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.194520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:53.194642 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.194672 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 07:39:53.209457 (+ 14785us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 07:39:53.209640 (+ 183us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.9.126:36395: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":367,"threads_started":1}
W20260926 07:39:53.210027 16623 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260926 07:39:53.211915 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16554
I20260926 07:39:53.212988 16615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:53.304337 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16554
2026-09-26T07:39:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 07:39:53.358903 11301 mini_oidc.cc:152] Starting JWKS server
I20260926 07:39:53.360663 11301 webserver.cc:460] Webserver started at https://127.0.0.1:46539/ using document root <none> and password file <none>
I20260926 07:39:53.360854 11301 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 07:39:53.361474 11301 webserver.cc:460] Webserver started at http://127.0.0.1:44249/ using document root <none> and password file <none>
2026-09-26T07:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:53Z Disabled control of system clock
I20260926 07:39:53.386029 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34233
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:44149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:34233
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46539/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 07:39:53.695921 16657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:53.696238 16657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:53.696395 16657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:53.703477 16657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:53.703608 16657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:53.703698 16657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:53.703812 16657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:53.703897 16657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 07:39:53.703992 16657 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 07:39:53.704070 16657 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46539/jwks.json
I20260926 07:39:53.712787 16657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:34233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46539/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:53.714039 16657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:53.715758 16657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:53.727288 16662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:53.727339 16665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:53.727288 16663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:53.728026 16657 server_base.cc:1034] running on GCE node
I20260926 07:39:53.728736 16657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:53.730072 16657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:53.731261 16657 hybrid_clock.cc:648] HybridClock initialized: now 1790408393731212 us; error 36 us; skew 500 ppm
I20260926 07:39:53.731559 16657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:53.742471 16657 webserver.cc:460] Webserver started at http://127.11.9.126:46763/ using document root <none> and password file <none>
I20260926 07:39:53.743153 16657 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:53.743240 16657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:53.743513 16657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:53.746308 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "229456535bbd4817926b0d4b98e48401"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "5b2ba51f2e024f96062d55d659206201"
server_key_iv: "d03319a17995b9ba75c778603886c578"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.747017 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "229456535bbd4817926b0d4b98e48401"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "5b2ba51f2e024f96062d55d659206201"
server_key_iv: "d03319a17995b9ba75c778603886c578"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.751817 16657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:39:53.755425 16671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:53.756695 16657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:39:53.756829 16657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "229456535bbd4817926b0d4b98e48401"
format_stamp: "Formatted at 2026-09-26 07:39:53 on dist-test-slave-x492"
server_key: "5b2ba51f2e024f96062d55d659206201"
server_key_iv: "d03319a17995b9ba75c778603886c578"
server_key_version: "encryptionkey@0"
I20260926 07:39:53.756999 16657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:53.774660 16657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:53.775332 16657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:53.775573 16657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:53.792872 16657 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:34233
I20260926 07:39:53.792891 16722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:34233 every 8 connection(s)
I20260926 07:39:53.794144 16657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:53.797812 16723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:53.802253 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16657
I20260926 07:39:53.802479 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:53.803090 11301 external_mini_cluster.cc:1428] Setting key 71018f35042865bc2c077ffc730a482b
I20260926 07:39:53.808176 16723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401: Bootstrap starting.
I20260926 07:39:53.811642 16723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:53.812713 16723 log.cc:824] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:53.815716 16723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401: No bootstrap required, opened a new log
I20260926 07:39:53.816550 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:53.805008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.11.9.126:34233)
0926 07:39:53.805981 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:53.805992 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:53.806034 (+ 42us) server_negotiation.cc:407] Connection header received
0926 07:39:53.807631 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:53.807653 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:53.808077 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:53.808431 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:53.808953 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.810294 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.812172 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.812873 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.813013 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.814365 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:53.814372 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:53.814385 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:53.814878 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:53.814949 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:53.814968 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:53.815051 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:53.815298 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 07:39:53.815614 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":306,"threads_started":1}
I20260926 07:39:53.820308 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:39:53.824327 16723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } }
I20260926 07:39:53.824661 16723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:53.824729 16723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229456535bbd4817926b0d4b98e48401, State: Initialized, Role: FOLLOWER
I20260926 07:39:53.825254 16723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } }
I20260926 07:39:53.825440 16723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:53.825536 16723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:53.825655 16723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:53.827610 16723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } }
I20260926 07:39:53.828009 16723 leader_election.cc:302] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 229456535bbd4817926b0d4b98e48401; no voters:
I20260926 07:39:53.828539 16723 leader_election.cc:288] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:53.828678 16728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:53.829535 16728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [term 1 LEADER]: Becoming Leader. State: Replica: 229456535bbd4817926b0d4b98e48401, State: Running, Role: LEADER
I20260926 07:39:53.830291 16728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } }
I20260926 07:39:53.830803 16723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:53.833112 16730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 229456535bbd4817926b0d4b98e48401. Latest consensus state: current_term: 1 leader_uuid: "229456535bbd4817926b0d4b98e48401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } } }
I20260926 07:39:53.833549 16730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:53.834041 16729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "229456535bbd4817926b0d4b98e48401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229456535bbd4817926b0d4b98e48401" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34233 } } }
I20260926 07:39:53.834389 16729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:53.836076 16734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:53.841251 16734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:53.850313 16734 catalog_manager.cc:1347] Generated new cluster ID: 5695fa4902cf4cc299dc50531cf7f637
I20260926 07:39:53.850420 16734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:53.865237 16734 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:53.866976 16734 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:53.890003 16734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 229456535bbd4817926b0d4b98e48401: Generated new TSK 0
I20260926 07:39:53.890892 16734 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 07:39:53.918171 16752 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:53.909384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:34233 (local address 127.0.0.1:55080)
0926 07:39:53.910020 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:53.910036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:53.910155 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:53.910627 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:53.910630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:53.910639 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 07:39:53.910876 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:53.910883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.912309 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.912313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:53.913459 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:53.913472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.914494 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.914498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:53.914633 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.914665 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 07:39:53.917513 (+ 2848us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 07:39:53.917734 (+ 221us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.9.126:34233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":380,"threads_started":1}
I20260926 07:39:53.918896 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:53.909384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.11.9.126:34233)
0926 07:39:53.909577 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:53.909584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:53.910126 (+ 542us) server_negotiation.cc:407] Connection header received
0926 07:39:53.910335 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:53.910346 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:53.910448 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:53.910612 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0926 07:39:53.911025 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.912166 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.913597 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:53.914316 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:53.914453 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:53.914939 (+ 486us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 07:39:53.917271 (+ 2332us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 07:39:53.918591 (+ 1320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55080: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 07:39:53.919003 16726 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55080: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260926 07:39:53.922186 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16657
I20260926 07:39:53.923312 16718 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 07:39:53.934854 16753 webserver.cc:533] Webserver: sslize failed:
E20260926 07:39:53.934968 16753 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 07:39:54.020469 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16657
2026-09-26T07:39:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (716 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T07:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:54Z Disabled control of system clock
I20260926 07:39:54.100473 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45241
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:36965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45241
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/testchainca.cert with env {}
W20260926 07:39:54.409132 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:54.409435 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:54.409526 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:54.416577 16758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:54.416669 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:54.416705 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:54.416761 16758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:54.416805 16758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 07:39:54.416863 16758 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 07:39:54.416903 16758 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 07:39:54.426383 16758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:36965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45241
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:54.427536 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:54.428697 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:54.440091 16766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:54.440228 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:54.440235 16764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:54.440814 16758 server_base.cc:1034] running on GCE node
I20260926 07:39:54.441668 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:54.442970 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:54.444192 16758 hybrid_clock.cc:648] HybridClock initialized: now 1790408394444157 us; error 54 us; skew 500 ppm
I20260926 07:39:54.444519 16758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:39:54.446692 16758 webserver.cc:460] Webserver started at http://127.11.9.126:38009/ using document root <none> and password file <none>
I20260926 07:39:54.447374 16758 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:54.447525 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:54.447829 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:54.450579 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "79ea540084d841619b99b43813458a89"
format_stamp: "Formatted at 2026-09-26 07:39:54 on dist-test-slave-x492"
server_key: "9faebea449b7bad2e0ec32635dddfe8a"
server_key_iv: "fcbbce7e567028c2194f53da1738e45a"
server_key_version: "encryptionkey@0"
I20260926 07:39:54.451346 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "79ea540084d841619b99b43813458a89"
format_stamp: "Formatted at 2026-09-26 07:39:54 on dist-test-slave-x492"
server_key: "9faebea449b7bad2e0ec32635dddfe8a"
server_key_iv: "fcbbce7e567028c2194f53da1738e45a"
server_key_version: "encryptionkey@0"
I20260926 07:39:54.456125 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:39:54.459295 16772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:54.460565 16758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:39:54.460747 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "79ea540084d841619b99b43813458a89"
format_stamp: "Formatted at 2026-09-26 07:39:54 on dist-test-slave-x492"
server_key: "9faebea449b7bad2e0ec32635dddfe8a"
server_key_iv: "fcbbce7e567028c2194f53da1738e45a"
server_key_version: "encryptionkey@0"
I20260926 07:39:54.460963 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:54.493883 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:54.494737 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:54.495019 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:54.513123 16758 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45241
I20260926 07:39:54.513149 16823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45241 every 8 connection(s)
I20260926 07:39:54.514375 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:54.517324 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16758
I20260926 07:39:54.517529 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:54.518157 11301 external_mini_cluster.cc:1428] Setting key b584948e639d90f8cac6184977f7d4a0
I20260926 07:39:54.518157 16824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:54.531175 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:54.519905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.11.9.126:45241)
0926 07:39:54.520881 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:54.520892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:54.520936 (+ 44us) server_negotiation.cc:407] Connection header received
0926 07:39:54.522640 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:54.522660 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:54.523093 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:54.523398 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:54.523983 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.525337 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.527029 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.527641 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.527766 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:54.528971 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:54.528977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:54.528989 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:39:54.529613 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:54.529681 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:39:54.529700 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:54.529785 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:54.530001 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 07:39:54.530325 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":396,"threads_started":1}
I20260926 07:39:54.531535 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89: Bootstrap starting.
I20260926 07:39:54.534652 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:39:54.534885 16824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:54.535837 16824 log.cc:824] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:54.538659 16824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89: No bootstrap required, opened a new log
I20260926 07:39:54.545521 16824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } }
I20260926 07:39:54.545850 16824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:54.545955 16824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ea540084d841619b99b43813458a89, State: Initialized, Role: FOLLOWER
I20260926 07:39:54.546463 16824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } }
I20260926 07:39:54.546690 16824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:54.546847 16824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:54.547039 16824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:54.549042 16824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } }
I20260926 07:39:54.549535 16824 leader_election.cc:302] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ea540084d841619b99b43813458a89; no voters:
I20260926 07:39:54.550076 16824 leader_election.cc:288] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:54.550194 16829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:54.550846 16829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [term 1 LEADER]: Becoming Leader. State: Replica: 79ea540084d841619b99b43813458a89, State: Running, Role: LEADER
I20260926 07:39:54.551332 16829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } }
I20260926 07:39:54.552075 16824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:54.553820 16830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ea540084d841619b99b43813458a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } } }
I20260926 07:39:54.554601 16830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:54.559254 16831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ea540084d841619b99b43813458a89. Latest consensus state: current_term: 1 leader_uuid: "79ea540084d841619b99b43813458a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ea540084d841619b99b43813458a89" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45241 } } }
I20260926 07:39:54.559582 16831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:54.561872 16837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:54.567200 16837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:54.575127 16837 catalog_manager.cc:1347] Generated new cluster ID: e5ff259f27d5422490e0cf09f67d6b41
I20260926 07:39:54.575235 16837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:54.608974 16837 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:54.610858 16837 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:54.623018 16837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ea540084d841619b99b43813458a89: Generated new TSK 0
I20260926 07:39:54.624002 16837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:39:54.786921 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:54.646733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.11.9.126:45241)
0926 07:39:54.646947 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:54.646955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:54.647298 (+ 343us) server_negotiation.cc:407] Connection header received
0926 07:39:54.647532 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:54.647535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:54.647620 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:54.647739 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0926 07:39:54.648289 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.648972 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.651114 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.651528 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.651626 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:54.652279 (+ 653us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 07:39:54.786375 (+134096us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 07:39:54.786696 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 07:39:54.787075 16827 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 07:39:54.787413 16853 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:54.646560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45241 (local address 127.0.0.1:34364)
0926 07:39:54.647188 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:54.647208 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:54.647361 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:54.647789 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:54.647795 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:54.647808 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 07:39:54.648149 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:54.648161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.649114 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.649120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:54.650979 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:54.650994 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:54.651701 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:54.651709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:54.651911 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:54.651955 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 07:39:54.786681 (+134726us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 07:39:54.786998 (+ 317us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.9.126:45241: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":323,"threads_started":1}
I20260926 07:39:54.789397 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16758
I20260926 07:39:54.790493 16819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:54.894150 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16758
2026-09-26T07:39:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (853 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (395 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (418 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:55 dist-test-slave-x492 krb5kdc[16868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:55 dist-test-slave-x492 krb5kdc[16868](info): set up 2 sockets
Sep 26 07:39:55 dist-test-slave-x492 krb5kdc[16868](info): commencing operation
krb5kdc: starting...
W20260926 07:39:56.789712 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:39:56 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:39:56.988112 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38725
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:38725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:57.305652 16881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:57.305928 16881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:57.306102 16881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:57.313071 16881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:39:57.313179 16881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:57.313297 16881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:57.313390 16881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:39:57.313473 16881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:39:57.322764 16881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:38725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:57.323982 16881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:57.325313 16881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:57.336694 16889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:57.336771 16886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:57.336763 16887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:57.337565 16881 server_base.cc:1034] running on GCE node
I20260926 07:39:57.338294 16881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:57.339609 16881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:57.340840 16881 hybrid_clock.cc:648] HybridClock initialized: now 1790408397340806 us; error 61 us; skew 500 ppm
I20260926 07:39:57.341220 16881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:57 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408397, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:57.344365 16881 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:39:57.345865 16881 webserver.cc:460] Webserver started at http://127.11.9.126:46579/ using document root <none> and password file <none>
I20260926 07:39:57.346506 16881 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:57.346652 16881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:57.346952 16881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:57.349789 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "6a7877350bba4df7a4ecd6302a3f78e7"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "a41f854ad13cc4e91e1b47e7e24a5713"
server_key_iv: "7484460a61e04a8eeb47018d9727a5b8"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.350525 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "6a7877350bba4df7a4ecd6302a3f78e7"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "a41f854ad13cc4e91e1b47e7e24a5713"
server_key_iv: "7484460a61e04a8eeb47018d9727a5b8"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.355336 16881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:39:57.358762 16896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:57.359964 16881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:57.360137 16881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "6a7877350bba4df7a4ecd6302a3f78e7"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "a41f854ad13cc4e91e1b47e7e24a5713"
server_key_iv: "7484460a61e04a8eeb47018d9727a5b8"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.360329 16881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:57.385289 16881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:57.389036 16881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:57.389333 16881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:57.407570 16881 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:38725
I20260926 07:39:57.407593 16947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:38725 every 8 connection(s)
I20260926 07:39:57.408875 16881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:39:57.412448 16948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:39:57.417068 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16881
I20260926 07:39:57.417303 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:39:57.418108 11301 external_mini_cluster.cc:1428] Setting key 8e35af60fb16eec334316dcdc8607d39
I20260926 07:39:57.423763 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7: Bootstrap starting.
I20260926 07:39:57.427258 16948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7: Neither blocks nor log segments found. Creating new log.
I20260926 07:39:57.428424 16948 log.cc:824] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7: Log is configured to *not* fsync() on all Append() calls
I20260926 07:39:57.431181 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7: No bootstrap required, opened a new log
Sep 26 07:39:57 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:57.440325 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:57.420091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.11.9.126:38725)
0926 07:39:57.420889 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:57.420900 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:57.420939 (+ 39us) server_negotiation.cc:407] Connection header received
0926 07:39:57.422734 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:57.422756 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:57.423108 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:57.423416 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:57.425073 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:57.426417 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:57.428157 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:57.428936 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:57.429147 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:57.433424 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:57.433431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:57.433445 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:57.433480 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:57.436481 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:57.437288 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:57.437294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:57.437299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:57.437451 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:57.437857 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:57.437860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:57.437862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:57.438351 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:57.438567 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:57.438993 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 07:39:57.439345 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":403,"threads_started":1}
I20260926 07:39:57.440409 16948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } }
I20260926 07:39:57.441021 16948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:39:57.441120 16948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7877350bba4df7a4ecd6302a3f78e7, State: Initialized, Role: FOLLOWER
I20260926 07:39:57.441771 16948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } }
I20260926 07:39:57.442068 16948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:39:57.442238 16948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:39:57.442425 16948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:39:57.444900 16948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } }
I20260926 07:39:57.445458 16948 leader_election.cc:302] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a7877350bba4df7a4ecd6302a3f78e7; no voters:
I20260926 07:39:57.446023 16948 leader_election.cc:288] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:39:57.446110 16953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:39:57.446784 16953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [term 1 LEADER]: Becoming Leader. State: Replica: 6a7877350bba4df7a4ecd6302a3f78e7, State: Running, Role: LEADER
I20260926 07:39:57.447293 16953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } }
I20260926 07:39:57.448074 16948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:39:57.449788 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } } }
I20260926 07:39:57.450129 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:57.450340 16955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a7877350bba4df7a4ecd6302a3f78e7. Latest consensus state: current_term: 1 leader_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7877350bba4df7a4ecd6302a3f78e7" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38725 } } }
I20260926 07:39:57.450865 16955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7 [sys.catalog]: This master's current role is: LEADER
I20260926 07:39:57.461952 16958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:39:57.467432 16958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:39:57.478212 16958 catalog_manager.cc:1347] Generated new cluster ID: 79da8b4a6d9249a8bdf4784138a53474
I20260926 07:39:57.478317 16958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:39:57.499089 16958 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:39:57.500833 16958 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:39:57.515033 16958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a7877350bba4df7a4ecd6302a3f78e7: Generated new TSK 0
I20260926 07:39:57.515913 16958 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:39:57.580276 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:57.902478 16975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:57.902743 16975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:57.902835 16975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:57.909893 16975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:57.909986 16975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:57.910086 16975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:39:57.919091 16975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:57.920315 16975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:57.921537 16975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:57.935067 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:57.935472 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:57.935612 16983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:57.936251 16975 server_base.cc:1034] running on GCE node
I20260926 07:39:57.936844 16975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:57.937801 16975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:57.939035 16975 hybrid_clock.cc:648] HybridClock initialized: now 1790408397938977 us; error 50 us; skew 500 ppm
I20260926 07:39:57.939343 16975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:57 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408397, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:57.942663 16975 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:39:57.944161 16975 webserver.cc:460] Webserver started at http://127.11.9.65:40239/ using document root <none> and password file <none>
I20260926 07:39:57.944826 16975 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:57.944965 16975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:57.945327 16975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:57.948276 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "00f904754ce84fe6875fa9e4432e5bfe"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "82a990e4b4c5d4bc1a729fda6ac26319"
server_key_iv: "ad30c732f11cf5011e53bcfcbebb9ad3"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.949100 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "00f904754ce84fe6875fa9e4432e5bfe"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "82a990e4b4c5d4bc1a729fda6ac26319"
server_key_iv: "ad30c732f11cf5011e53bcfcbebb9ad3"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.954017 16975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:39:57.957330 16990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:57.958640 16975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:39:57.958801 16975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "00f904754ce84fe6875fa9e4432e5bfe"
format_stamp: "Formatted at 2026-09-26 07:39:57 on dist-test-slave-x492"
server_key: "82a990e4b4c5d4bc1a729fda6ac26319"
server_key_iv: "ad30c732f11cf5011e53bcfcbebb9ad3"
server_key_version: "encryptionkey@0"
I20260926 07:39:57.958993 16975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:57.982656 16975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:57.986661 16975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:57.986945 16975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:57.987851 16975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:57.989081 16975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:57.989207 16975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:57.989355 16975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:57.989437 16975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.022063 16975 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44687
I20260926 07:39:58.022118 17102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44687 every 8 connection(s)
I20260926 07:39:58.023380 16975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:39:58.031519 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 16975
I20260926 07:39:58.031716 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:39:58.032328 11301 external_mini_cluster.cc:1428] Setting key a883bace9eeffe963058b5f040e84933
Sep 26 07:39:58 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408397, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:58.044152 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:58.025971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:40997 (local address 127.11.9.126:38725)
0926 07:39:58.026387 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:58.026393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:58.027384 (+ 991us) server_negotiation.cc:407] Connection header received
0926 07:39:58.029072 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:58.029076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:58.029140 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:58.029291 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:58.030687 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.031663 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.033365 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.033921 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.034069 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:58.038028 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:58.038040 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:58.038044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:58.038073 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:58.041001 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:58.041735 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:58.041743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:58.041750 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:58.041890 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:58.042275 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:58.042283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:58.042289 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:58.042644 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:58.042787 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:58.043435 (+ 648us) server_negotiation.cc:299] Negotiation successful
0926 07:39:58.043665 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":199,"threads_started":1}
I20260926 07:39:58.044797 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:58.026545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38725 (local address 127.11.9.65:40997)
0926 07:39:58.027233 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:58.027269 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:58.028825 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:58.029483 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:58.029491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:58.029984 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:58.030538 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:58.030554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.031842 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.031848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:58.033215 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:58.033237 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.034156 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.034161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:58.034463 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:58.035164 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:58.035202 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:58.037830 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:58.041143 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:58.041149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:58.041161 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:58.041612 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:58.041995 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:58.041998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:58.042000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:58.042166 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:58.042767 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:58.042773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:58.043063 (+ 290us) client_negotiation.cc:769] Sending connection context
0926 07:39:58.043279 (+ 216us) client_negotiation.cc:240] Negotiation successful
0926 07:39:58.043716 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":276,"threads_started":1}
I20260926 07:39:58.046329 17103 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38725
I20260926 07:39:58.046646 17103 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:58.047330 17103 heartbeater.cc:511] Master 127.11.9.126:38725 requested a full tablet report, sending...
I20260926 07:39:58.049484 16913 ts_manager.cc:194] Registered new tserver with Master: 00f904754ce84fe6875fa9e4432e5bfe (127.11.9.65:44687)
I20260926 07:39:58.050854 16913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:40997
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:39:58.076481 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:58.389344 17109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:58.389654 17109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:58.389823 17109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:58.397002 17109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:58.397143 17109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:58.397365 17109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:39:58.407660 17109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:58.408908 17109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:58.410229 17109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:58.424669 17114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:58.424633 17115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:58.425110 17117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:58.425951 17109 server_base.cc:1034] running on GCE node
I20260926 07:39:58.426573 17109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:58.427477 17109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:58.428670 17109 hybrid_clock.cc:648] HybridClock initialized: now 1790408398428607 us; error 55 us; skew 500 ppm
I20260926 07:39:58.428972 17109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:58 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408398, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:58.432080 17109 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:39:58.433559 17109 webserver.cc:460] Webserver started at http://127.11.9.66:40839/ using document root <none> and password file <none>
I20260926 07:39:58.434228 17109 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:58.434358 17109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:58.434660 17109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:58.437598 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "8613fafcf80a4b908dfe1c87b7f71876"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "4c93de581123a3934d4c90aa0a2b8725"
server_key_iv: "d4222e40bf40471c248e945591fccb2f"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.438355 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "8613fafcf80a4b908dfe1c87b7f71876"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "4c93de581123a3934d4c90aa0a2b8725"
server_key_iv: "d4222e40bf40471c248e945591fccb2f"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.443289 17109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260926 07:39:58.446551 17124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.447855 17109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:39:58.448068 17109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "8613fafcf80a4b908dfe1c87b7f71876"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "4c93de581123a3934d4c90aa0a2b8725"
server_key_iv: "d4222e40bf40471c248e945591fccb2f"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.448249 17109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:58.463856 17109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:58.468343 17109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:58.468624 17109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:58.469547 17109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:58.470744 17109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:58.470816 17109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.470933 17109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:58.470986 17109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.502705 17109 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45653
I20260926 07:39:58.502748 17236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45653 every 8 connection(s)
I20260926 07:39:58.503983 17109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:39:58.514362 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17109
I20260926 07:39:58.514552 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:39:58.515168 11301 external_mini_cluster.cc:1428] Setting key 66b9f4723b0989b96766ba802001ad0f
Sep 26 07:39:58 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408398, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:58.522902 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:58.506617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:35097 (local address 127.11.9.126:38725)
0926 07:39:58.506778 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:58.506783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:58.507854 (+ 1071us) server_negotiation.cc:407] Connection header received
0926 07:39:58.509646 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:58.509652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:58.509743 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:58.509850 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:58.511231 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.511930 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.513519 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.513925 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.514057 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:58.517666 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:58.517671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:58.517674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:58.517703 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:58.520048 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:58.520720 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:58.520732 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:58.520735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:58.520854 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:58.521317 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:58.521322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:58.521325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:58.521574 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:58.521735 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:58.522366 (+ 631us) server_negotiation.cc:299] Negotiation successful
0926 07:39:58.522576 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:39:58.523815 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:58.507005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38725 (local address 127.11.9.66:35097)
0926 07:39:58.507702 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:58.507738 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:58.509393 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:58.510064 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:58.510072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:58.510535 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:58.511064 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:58.511080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.512119 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.512123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:58.513384 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:58.513395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:58.514149 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:58.514157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:58.514375 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:58.515385 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:58.515406 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:58.517446 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:58.520180 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:58.520190 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:58.520206 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:58.520590 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:58.520960 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:58.520967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:58.520970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:58.521150 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:58.521699 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:58.521709 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:58.522020 (+ 311us) client_negotiation.cc:769] Sending connection context
0926 07:39:58.522268 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 07:39:58.522760 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":280,"threads_started":1}
I20260926 07:39:58.525274 17237 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38725
I20260926 07:39:58.525566 17237 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:58.526274 17237 heartbeater.cc:511] Master 127.11.9.126:38725 requested a full tablet report, sending...
I20260926 07:39:58.527704 16913 ts_manager.cc:194] Registered new tserver with Master: 8613fafcf80a4b908dfe1c87b7f71876 (127.11.9.66:45653)
I20260926 07:39:58.529011 16913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:35097
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:39:58.560132 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:39:58.875905 17242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:39:58.876183 17242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:39:58.876274 17242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:39:58.883373 17242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:39:58.883469 17242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:39:58.883577 17242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:39:58.892831 17242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:39:58.894047 17242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:39:58.895262 17242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:39:58.907927 17247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:58.907976 17250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:39:58.907891 17248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:39:58.908841 17242 server_base.cc:1034] running on GCE node
I20260926 07:39:58.909463 17242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:39:58.910313 17242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:39:58.911530 17242 hybrid_clock.cc:648] HybridClock initialized: now 1790408398911496 us; error 49 us; skew 500 ppm
I20260926 07:39:58.911823 17242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:39:58 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408398, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:58.918370 17242 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:39:58.919699 17242 webserver.cc:460] Webserver started at http://127.11.9.67:41305/ using document root <none> and password file <none>
I20260926 07:39:58.920353 17242 fs_manager.cc:359] Metadata directory not provided
I20260926 07:39:58.920501 17242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:39:58.920797 17242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:39:58.923566 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "3b37de6e3e0e4a29863b35cfafa08897"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "006f4b0a7812d760c256ac808fa857aa"
server_key_iv: "af0adf3a3db4afe8e0b6a4726147f052"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.924311 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "3b37de6e3e0e4a29863b35cfafa08897"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "006f4b0a7812d760c256ac808fa857aa"
server_key_iv: "af0adf3a3db4afe8e0b6a4726147f052"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.929188 17242 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 07:39:58.932555 17257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.933949 17242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:39:58.934110 17242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "3b37de6e3e0e4a29863b35cfafa08897"
format_stamp: "Formatted at 2026-09-26 07:39:58 on dist-test-slave-x492"
server_key: "006f4b0a7812d760c256ac808fa857aa"
server_key_iv: "af0adf3a3db4afe8e0b6a4726147f052"
server_key_version: "encryptionkey@0"
I20260926 07:39:58.934305 17242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:39:58.957602 17242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:39:58.961313 17242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:39:58.961584 17242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:39:58.962569 17242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:39:58.964154 17242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:39:58.964260 17242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.964406 17242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:39:58.964476 17242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:39:58.996939 17242 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:33605
I20260926 07:39:58.996982 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:33605 every 8 connection(s)
I20260926 07:39:58.998261 17242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:39:59.003754 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17242
I20260926 07:39:59.003916 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:39:59.004426 11301 external_mini_cluster.cc:1428] Setting key 2a4561205238fd4ae87c86aaa5827d80
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408398, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:39:59.017787 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.000766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:35785 (local address 127.11.9.126:38725)
0926 07:39:59.000960 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.000967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.002245 (+ 1278us) server_negotiation.cc:407] Connection header received
0926 07:39:59.004120 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.004126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.004217 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.004333 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.006071 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.007081 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.008695 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.009076 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.009223 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.012601 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.012606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.012609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.012639 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.015020 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.015675 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.015679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.015681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.015792 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.016156 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.016169 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.016170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.016405 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.016509 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.017294 (+ 785us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.017449 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:39:59.019052 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.001374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38725 (local address 127.11.9.67:35785)
0926 07:39:59.002076 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0926 07:39:59.002120 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:39:59.003753 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:39:59.004570 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:39:59.004579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:39:59.005115 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:39:59.005902 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:59.005920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.007287 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.007293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:59.008572 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:39:59.008584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.009317 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.009322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:39:59.009524 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.010198 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:39:59.010224 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:39:59.012416 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:39:59.015144 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:59.015151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:59.015164 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:59.015571 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:59.015880 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:39:59.015884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:39:59.015886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:39:59.016058 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:39:59.016524 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:39:59.016542 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:39:59.016914 (+ 372us) client_negotiation.cc:769] Sending connection context
0926 07:39:59.017140 (+ 226us) client_negotiation.cc:240] Negotiation successful
0926 07:39:59.017624 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":294,"threads_started":1}
I20260926 07:39:59.020445 17370 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38725
I20260926 07:39:59.020718 17370 heartbeater.cc:464] Registering TS with master...
I20260926 07:39:59.021564 17370 heartbeater.cc:511] Master 127.11.9.126:38725 requested a full tablet report, sending...
I20260926 07:39:59.023141 16913 ts_manager.cc:194] Registered new tserver with Master: 3b37de6e3e0e4a29863b35cfafa08897 (127.11.9.67:33605)
I20260926 07:39:59.024289 16913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:35785
I20260926 07:39:59.032223 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:39:59.053674 17103 heartbeater.cc:503] Master 127.11.9.126:38725 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/test-admin.keytab.
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.058717 11301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 07:39:59.063760 17377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:59.061197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.11.9.65:44687)
0926 07:39:59.061891 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.061900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.061930 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:39:59.062000 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.062006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.062215 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.062377 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.063284 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42522: BlockingRecv error: recv got EOF from 127.0.0.1:42522 (error 108)
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":338,"threads_started":1}
W20260926 07:39:59.063836 11301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc' of type FILE
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.065477 11301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.066792 11301 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:59.081319 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.068320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.11.9.65:44687)
0926 07:39:59.068539 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.068546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.068759 (+ 213us) server_negotiation.cc:407] Connection header received
0926 07:39:59.068943 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.068948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.069011 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.069105 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.070096 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.071022 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.072216 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.072777 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.072905 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.076166 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.076175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.076189 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.076218 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.078694 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.079312 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.079319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.079321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.079429 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.079794 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.079800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.079804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.080124 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.080416 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.080741 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.080976 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260926 07:39:59.086644 17377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:59.085054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.11.9.65:44687)
0926 07:39:59.085296 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.085302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.085520 (+ 218us) server_negotiation.cc:407] Connection header received
0926 07:39:59.085624 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.085627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.085678 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.085761 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.086419 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42528: BlockingRecv error: recv got EOF from 127.0.0.1:42528 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260926 07:39:59.086835 11301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc' of type FILE
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.088227 11301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.089548 11301 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:59.102468 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.090650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.11.9.65:44687)
0926 07:39:59.090797 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.090801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.091166 (+ 365us) server_negotiation.cc:407] Connection header received
0926 07:39:59.091276 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.091280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.091328 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.091404 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.092259 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.092902 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.094083 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.094496 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.094587 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.097881 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.097886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.097889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.097925 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.100214 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.100788 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.100791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.100793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.100899 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.101236 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.101239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.101241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.101472 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.101663 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.102012 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.102153 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 07:39:59.106559 17377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:39:59.104800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.11.9.65:44687)
0926 07:39:59.104969 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.104977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.105344 (+ 367us) server_negotiation.cc:407] Connection header received
0926 07:39:59.105521 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.105526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.105596 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.105691 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.106338 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42544: BlockingRecv error: recv got EOF from 127.0.0.1:42544 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 07:39:59.106724 11301 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790408351541924-11301-0/krb5kdc/krb5cc' of type FILE
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.108115 11301 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:39:59.109445 11301 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:39:59.122766 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.110540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.11.9.65:44687)
0926 07:39:59.110727 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.110735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.110983 (+ 248us) server_negotiation.cc:407] Connection header received
0926 07:39:59.111154 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.111172 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.111219 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.111313 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.112216 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.112890 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.114188 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.114592 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.114719 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.118022 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.118032 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.118035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.118062 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.120316 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.120956 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.120963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.120967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.121094 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.121489 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.121495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.121498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.121737 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.122039 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.122300 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.122488 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:39:59.124141 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16975
I20260926 07:39:59.134800 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.124772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.11.9.65:44687)
0926 07:39:59.124962 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.124968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.125208 (+ 240us) server_negotiation.cc:407] Connection header received
0926 07:39:59.125343 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.125346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.125393 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.125460 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.126266 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.126871 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.127954 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.128334 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.128418 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.130356 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.130360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.130364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.130389 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.132517 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.133052 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.133056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.133058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.133229 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.133515 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.133517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.133519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.133732 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.133922 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.134321 (+ 399us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.134489 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:39:59.135342 17098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:59.217938 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16975
I20260926 07:39:59.233808 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17109
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:39:59.248502 17390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.234658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.11.9.66:45653)
0926 07:39:59.235167 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.235175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.235201 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:39:59.235260 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.235266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.235473 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.235597 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.236474 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.237538 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.238809 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.239366 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.239469 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.243007 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.243016 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.243029 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.243057 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.245713 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.246458 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.246465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.246467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.246581 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.246962 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.246968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.246971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.247292 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.247614 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.247955 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.248199 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":263,"threads_started":1}
I20260926 07:39:59.249922 17232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:59.330245 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17109
I20260926 07:39:59.345847 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17242
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:39:59.360880 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:39:59.346764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.11.9.67:33605)
0926 07:39:59.347383 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0926 07:39:59.347390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:39:59.347419 (+ 29us) server_negotiation.cc:407] Connection header received
0926 07:39:59.347475 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:39:59.347482 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:39:59.347695 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:39:59.347817 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:39:59.348919 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.350065 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.351222 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:39:59.351748 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:39:59.351838 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:39:59.356073 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:39:59.356084 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:39:59.356099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:39:59.356134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:39:59.358723 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.359311 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.359318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.359320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.359429 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:39:59.359734 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:39:59.359737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:39:59.359739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:39:59.360035 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:39:59.360262 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 07:39:59.360436 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 07:39:59.360612 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":297,"threads_started":1}
I20260926 07:39:59.362294 17365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:59.443971 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17242
I20260926 07:39:59.459486 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 16881
I20260926 07:39:59.460592 16943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:39:59.562060 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 16881
2026-09-26T07:39:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3859 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[17397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[17397](info): set up 2 sockets
Sep 26 07:39:59 dist-test-slave-x492 krb5kdc[17397](info): commencing operation
krb5kdc: starting...
W20260926 07:40:00.653949 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:00 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab.
Entry for principal oryx/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab.
I20260926 07:40:00.853049 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36123
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab
--principal=oryx/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:36123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:01.171355 17410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:01.171634 17410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:01.171801 17410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:01.179311 17410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:01.179417 17410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:01.179528 17410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:01.179638 17410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:01.179721 17410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:01.189338 17410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:36123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab
--principal=oryx/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:36123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:01.190670 17410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:01.191975 17410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:01.203666 17418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:01.203737 17416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:01.203639 17415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:01.210795 17410 server_base.cc:1034] running on GCE node
I20260926 07:40:01.211689 17410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:01.213145 17410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:01.214434 17410 hybrid_clock.cc:648] HybridClock initialized: now 1790408401214384 us; error 45 us; skew 500 ppm
I20260926 07:40:01.214771 17410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:01 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408401, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:01.218183 17410 init.cc:377] Logged in from keytab as oryx/127.11.9.126@KRBTEST.COM (short username oryx)
I20260926 07:40:01.219678 17410 webserver.cc:460] Webserver started at http://127.11.9.126:35899/ using document root <none> and password file <none>
I20260926 07:40:01.220398 17410 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:01.220541 17410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:01.220847 17410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:01.224036 17410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "43e5257e3e904e598200dcaf9445dba3"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "8f8c4efb0c770a11e84d7ed7e5bb40ae"
server_key_iv: "4c6909e7f6ec9050d3123857283ec581"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.224812 17410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "43e5257e3e904e598200dcaf9445dba3"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "8f8c4efb0c770a11e84d7ed7e5bb40ae"
server_key_iv: "4c6909e7f6ec9050d3123857283ec581"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.229978 17410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 07:40:01.233319 17425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:01.234575 17410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:01.234766 17410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "43e5257e3e904e598200dcaf9445dba3"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "8f8c4efb0c770a11e84d7ed7e5bb40ae"
server_key_iv: "4c6909e7f6ec9050d3123857283ec581"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.234977 17410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:01.264391 17410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:01.268523 17410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:01.268826 17410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:01.286831 17410 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:36123
I20260926 07:40:01.286861 17476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:36123 every 8 connection(s)
I20260926 07:40:01.288177 17410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:01.291472 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17410
I20260926 07:40:01.291690 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:01.291941 17477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:01.292456 11301 external_mini_cluster.cc:1428] Setting key a5a664d1265d203bc26754fdcf916a84
I20260926 07:40:01.306597 17477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Bootstrap starting.
Sep 26 07:40:01 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:01.309645 17477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:01.310680 17477 log.cc:824] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:01.313586 17477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: No bootstrap required, opened a new log
I20260926 07:40:01.315794 17480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:01.294395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.11.9.126:36123)
0926 07:40:01.295240 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:01.295256 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:01.295320 (+ 64us) server_negotiation.cc:407] Connection header received
0926 07:40:01.297309 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:01.297334 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:01.297719 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:01.298039 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:01.299720 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.300936 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.302634 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.303419 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.303548 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:01.308083 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:01.308091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:01.308104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:01.308147 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:01.311237 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:01.312043 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:01.312051 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:01.312065 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:01.312217 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:01.312605 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:01.312618 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:01.312620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:01.313097 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:01.313437 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:01.314041 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 07:40:01.314506 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":414,"threads_started":1}
I20260926 07:40:01.321676 17477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } }
I20260926 07:40:01.322012 17477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:01.322130 17477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43e5257e3e904e598200dcaf9445dba3, State: Initialized, Role: FOLLOWER
I20260926 07:40:01.322762 17477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } }
I20260926 07:40:01.323007 17477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:01.323149 17477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:01.323325 17477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:01.325421 17477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } }
I20260926 07:40:01.326004 17477 leader_election.cc:302] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43e5257e3e904e598200dcaf9445dba3; no voters:
I20260926 07:40:01.326543 17477 leader_election.cc:288] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:01.326712 17482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:01.327370 17482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 LEADER]: Becoming Leader. State: Replica: 43e5257e3e904e598200dcaf9445dba3, State: Running, Role: LEADER
I20260926 07:40:01.328279 17482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } }
I20260926 07:40:01.328706 17477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:01.330871 17484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e5257e3e904e598200dcaf9445dba3. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } }
I20260926 07:40:01.331214 17484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:01.331748 17483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } }
I20260926 07:40:01.332078 17483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:01.332206 17488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:01.344585 17488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:01.354354 17488 catalog_manager.cc:1347] Generated new cluster ID: 65811b5f4021461f985a334f6a402d4e
I20260926 07:40:01.354475 17488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:01.368206 17488 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:01.370057 17488 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:01.383706 17488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Generated new TSK 0
I20260926 07:40:01.384791 17488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
I20260926 07:40:01.439237 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:01.756191 17504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:01.756500 17504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:01.756673 17504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:01.763742 17504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:01.763875 17504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:01.764060 17504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:01.773612 17504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:01.774820 17504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:01.776070 17504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:01.788801 17509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:01.788887 17512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:01.788913 17510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:01.790006 17504 server_base.cc:1034] running on GCE node
I20260926 07:40:01.790572 17504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:01.791442 17504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:01.792629 17504 hybrid_clock.cc:648] HybridClock initialized: now 1790408401792590 us; error 29 us; skew 500 ppm
I20260926 07:40:01.792888 17504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:01 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408401, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:01.795948 17504 init.cc:377] Logged in from keytab as oryx/127.11.9.65@KRBTEST.COM (short username oryx)
I20260926 07:40:01.797297 17504 webserver.cc:460] Webserver started at http://127.11.9.65:45721/ using document root <none> and password file <none>
I20260926 07:40:01.798033 17504 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:01.798164 17504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:01.798496 17504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:01.801496 17504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "9d656517c6e145fb8bfeacddd154fd03"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "52e01e517d16a41d472b063042895e56"
server_key_iv: "48a26828b68bc429593d46a2c795f81e"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.802242 17504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "9d656517c6e145fb8bfeacddd154fd03"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "52e01e517d16a41d472b063042895e56"
server_key_iv: "48a26828b68bc429593d46a2c795f81e"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.807278 17504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:40:01.810568 17519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:01.811841 17504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:40:01.812014 17504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "9d656517c6e145fb8bfeacddd154fd03"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "52e01e517d16a41d472b063042895e56"
server_key_iv: "48a26828b68bc429593d46a2c795f81e"
server_key_version: "encryptionkey@0"
I20260926 07:40:01.812201 17504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:01.833825 17504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:01.837617 17504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:01.837894 17504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:01.838915 17504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:01.840461 17504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:01.840564 17504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:01.840703 17504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:01.840780 17504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:01.873138 17504 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:43457
I20260926 07:40:01.873250 17631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:43457 every 8 connection(s)
I20260926 07:40:01.874437 17504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:01.880048 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17504
I20260926 07:40:01.880270 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:01.880882 11301 external_mini_cluster.cc:1428] Setting key 78ca347b573c8e376d012c1a68a3747c
Sep 26 07:40:01 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408401, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:01.893491 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:01.876790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:39387 (local address 127.11.9.126:36123)
0926 07:40:01.877281 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:01.877286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:01.878079 (+ 793us) server_negotiation.cc:407] Connection header received
0926 07:40:01.879853 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:01.879859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:01.879947 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:01.880065 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:01.881724 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.882644 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.884171 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.884577 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.884706 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:01.888238 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:01.888243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:01.888246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:01.888274 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:01.890631 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:01.891333 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:01.891342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:01.891345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:01.891449 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:01.891859 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:01.891866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:01.891873 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:01.892119 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:01.892274 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:01.892900 (+ 626us) server_negotiation.cc:299] Negotiation successful
0926 07:40:01.893099 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":240,"threads_started":1}
I20260926 07:40:01.894379 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:01.877231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:39387)
0926 07:40:01.877924 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:01.877961 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:01.879575 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:01.880306 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:01.880314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:01.880840 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:01.881515 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:01.881542 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.882786 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.882793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:01.884025 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:01.884036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:01.884786 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:01.884794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:01.885036 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:01.885742 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:01.885765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:01.888035 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:01.890777 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:01.890787 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:01.890804 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:01.891189 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:01.891553 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:01.891559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:01.891562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:01.891733 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:01.892248 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:01.892255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:01.892567 (+ 312us) client_negotiation.cc:769] Sending connection context
0926 07:40:01.892813 (+ 246us) client_negotiation.cc:240] Negotiation successful
0926 07:40:01.893322 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260926 07:40:01.895812 17632 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:01.896148 17632 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:01.897029 17632 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:01.899590 17442 ts_manager.cc:194] Registered new tserver with Master: 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457)
WARNING: no policy specified for oryx/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:40:01.901414 17442 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:39387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
I20260926 07:40:01.929056 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:02.249913 17638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:02.250240 17638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:02.250444 17638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:02.257493 17638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:02.257632 17638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:02.257819 17638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:02.267354 17638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:02.268635 17638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:02.269949 17638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:02.282819 17643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:02.282776 17646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:02.282855 17644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:02.283553 17638 server_base.cc:1034] running on GCE node
I20260926 07:40:02.284160 17638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:02.285071 17638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:02.286309 17638 hybrid_clock.cc:648] HybridClock initialized: now 1790408402286259 us; error 42 us; skew 500 ppm
I20260926 07:40:02.286640 17638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:02 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408402, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:02.290189 17638 init.cc:377] Logged in from keytab as oryx/127.11.9.66@KRBTEST.COM (short username oryx)
I20260926 07:40:02.294440 17638 webserver.cc:460] Webserver started at http://127.11.9.66:35331/ using document root <none> and password file <none>
I20260926 07:40:02.295116 17638 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:02.295267 17638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:02.295575 17638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:02.298604 17638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "22756c7c2632488bb848bd512965b705"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e1bab7a5ce7ee0485cbd42a12f5710a1"
server_key_iv: "9767f7bc4e238a8eff4bc27ff354a505"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.299366 17638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "22756c7c2632488bb848bd512965b705"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e1bab7a5ce7ee0485cbd42a12f5710a1"
server_key_iv: "9767f7bc4e238a8eff4bc27ff354a505"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.304349 17638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 07:40:02.307900 17653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.309230 17638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:40:02.309408 17638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "22756c7c2632488bb848bd512965b705"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e1bab7a5ce7ee0485cbd42a12f5710a1"
server_key_iv: "9767f7bc4e238a8eff4bc27ff354a505"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.309612 17638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:02.329840 17638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:02.333542 17638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:02.333830 17638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:02.334779 17638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:02.336151 17638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:02.336265 17638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.336419 17638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:02.336498 17638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.369446 17638 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:43065
I20260926 07:40:02.369489 17765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:43065 every 8 connection(s)
I20260926 07:40:02.370745 17638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:02.377880 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17638
I20260926 07:40:02.378046 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:02.378618 11301 external_mini_cluster.cc:1428] Setting key cb909d8fe454ca627697688b057d3a8b
Sep 26 07:40:02 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408402, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:02.391461 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:02.373377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:43933 (local address 127.11.9.126:36123)
0926 07:40:02.373546 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:02.373551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:02.374701 (+ 1150us) server_negotiation.cc:407] Connection header received
0926 07:40:02.376476 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:02.376480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:02.376546 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:02.376634 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:02.378267 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.379385 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.381064 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.381813 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.381961 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:02.385316 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:02.385325 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:02.385332 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:02.385376 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:02.388348 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:02.389024 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:02.389032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:02.389038 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:02.389218 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:02.389606 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:02.389613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:02.389620 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:02.389997 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:02.390150 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:02.390777 (+ 627us) server_negotiation.cc:299] Negotiation successful
0926 07:40:02.391040 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:40:02.392189 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:02.373801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:43933)
0926 07:40:02.374533 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:02.374571 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:02.376220 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:02.376842 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:02.376850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:02.377426 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:02.378053 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:02.378072 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.379539 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.379543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:02.380925 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:02.380936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.382005 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.382010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:02.382196 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:02.382878 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:02.382899 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:02.385076 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:02.388467 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:02.388473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:02.388486 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:02.388895 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:02.389324 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:02.389327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:02.389329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:02.389493 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:02.390110 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:02.390115 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:02.390411 (+ 296us) client_negotiation.cc:769] Sending connection context
0926 07:40:02.390626 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 07:40:02.391010 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":285,"threads_started":1}
I20260926 07:40:02.393642 17766 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:02.393966 17766 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:02.394822 17766 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:02.396407 17442 ts_manager.cc:194] Registered new tserver with Master: 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:02.397289 17442 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:43933
WARNING: no policy specified for oryx/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
I20260926 07:40:02.427879 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:02.739540 17771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:02.739857 17771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:02.739954 17771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:02.747069 17771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:02.747169 17771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:02.747273 17771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:02.756531 17771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:02.757683 17771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:02.758845 17771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:02.772439 17777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:02.772475 17779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:02.772498 17776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:02.773268 17771 server_base.cc:1034] running on GCE node
I20260926 07:40:02.773861 17771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:02.774837 17771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:02.776180 17771 hybrid_clock.cc:648] HybridClock initialized: now 1790408402776121 us; error 46 us; skew 500 ppm
I20260926 07:40:02.776482 17771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:02 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408402, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:02.779489 17771 init.cc:377] Logged in from keytab as oryx/127.11.9.67@KRBTEST.COM (short username oryx)
I20260926 07:40:02.780774 17771 webserver.cc:460] Webserver started at http://127.11.9.67:44811/ using document root <none> and password file <none>
I20260926 07:40:02.781482 17771 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:02.781621 17771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:02.781908 17771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:02.784613 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "cdd386c92a9d438087d817da3da7d399"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e647973adb8f04beee3e4c464ca67a42"
server_key_iv: "e63622261055dd164d0933ff5e5b08ca"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.785535 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "cdd386c92a9d438087d817da3da7d399"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e647973adb8f04beee3e4c464ca67a42"
server_key_iv: "e63622261055dd164d0933ff5e5b08ca"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.790416 17771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 07:40:02.793794 17786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.795019 17771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:40:02.795194 17771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "cdd386c92a9d438087d817da3da7d399"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e647973adb8f04beee3e4c464ca67a42"
server_key_iv: "e63622261055dd164d0933ff5e5b08ca"
server_key_version: "encryptionkey@0"
I20260926 07:40:02.795401 17771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:02.818142 17771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:02.822284 17771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:02.822574 17771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:02.823496 17771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:02.824766 17771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:02.824870 17771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.825014 17771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:02.825085 17771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:02.857693 17771 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:36999
I20260926 07:40:02.857777 17898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:36999 every 8 connection(s)
I20260926 07:40:02.858973 17771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:02.866592 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17771
I20260926 07:40:02.866775 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:02.867265 11301 external_mini_cluster.cc:1428] Setting key cc6dbd10f1a52e94c414666c668c5068
Sep 26 07:40:02 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408402, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:02.879036 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:02.861405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:51313 (local address 127.11.9.126:36123)
0926 07:40:02.861614 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:02.861620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:02.862811 (+ 1191us) server_negotiation.cc:407] Connection header received
0926 07:40:02.864492 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:02.864496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:02.864558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:02.864644 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:02.866076 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.867195 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.869077 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.869573 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.869666 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:02.873102 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:02.873110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:02.873114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:02.873152 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:02.876287 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:02.876957 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:02.876963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:02.876967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:02.877107 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:02.877588 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:02.877594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:02.877596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:02.877930 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:02.878059 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:02.878649 (+ 590us) server_negotiation.cc:299] Negotiation successful
0926 07:40:02.878811 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:40:02.879900 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:02.861957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:51313)
0926 07:40:02.862660 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:02.862697 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:02.864247 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:02.864843 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:02.864851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:02.865415 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:02.865938 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:02.865955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.867377 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.867381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:02.868932 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:02.868949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:02.869783 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:02.869791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:02.870077 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:02.870664 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:02.870684 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:02.872902 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:02.876427 (+ 3525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:02.876433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:02.876443 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:02.876834 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:02.877238 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:02.877241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:02.877243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:02.877448 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:02.878058 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:02.878064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:02.878328 (+ 264us) client_negotiation.cc:769] Sending connection context
0926 07:40:02.878555 (+ 227us) client_negotiation.cc:240] Negotiation successful
0926 07:40:02.878923 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":280,"threads_started":1}
I20260926 07:40:02.881294 17899 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:02.881615 17899 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:02.882419 17899 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:02.883634 17442 ts_manager.cc:194] Registered new tserver with Master: cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
I20260926 07:40:02.884413 17442 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:51313
I20260926 07:40:02.894357 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:40:02.904402 17632 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.11.9.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.125.keytab.
Entry for principal oryx/127.11.9.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.125.keytab.
I20260926 07:40:02.940263 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:32823
--webserver_interface=127.11.9.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.125.keytab
--principal=oryx/127.11.9.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:36123,127.11.9.125:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:03.260000 17904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:03.260247 17904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:03.260462 17904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:03.267738 17904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:03.267848 17904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:03.267916 17904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:03.268010 17904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:03.268085 17904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:03.277539 17904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:36123,127.11.9.125:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.125.keytab
--principal=oryx/127.11.9.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.9.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
W20260926 07:40:03.277889 17904 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.9.126:36123,127.11.9.125:32823'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 07:40:03.278851 17904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:03.280112 17904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:03.291476 17910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:03.291512 17912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:03.291947 17909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:03.293138 17904 server_base.cc:1034] running on GCE node
I20260926 07:40:03.293893 17904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:03.295195 17904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:03.296442 17904 hybrid_clock.cc:648] HybridClock initialized: now 1790408403296372 us; error 40 us; skew 500 ppm
I20260926 07:40:03.296766 17904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:03.299868 17904 init.cc:377] Logged in from keytab as oryx/127.11.9.125@KRBTEST.COM (short username oryx)
I20260926 07:40:03.301332 17904 webserver.cc:460] Webserver started at http://127.11.9.125:36647/ using document root <none> and password file <none>
I20260926 07:40:03.302054 17904 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:03.302177 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:03.302487 17904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:03.305348 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/instance:
uuid: "7585b7acbba641868d271945a613d6b5"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "2908bea76e60f1fe7481203eb8a51b31"
server_key_iv: "cbbe812302af5e59e9f4ce80026f023f"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.306113 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal/instance:
uuid: "7585b7acbba641868d271945a613d6b5"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "2908bea76e60f1fe7481203eb8a51b31"
server_key_iv: "cbbe812302af5e59e9f4ce80026f023f"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.310950 17904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:40:03.314221 17919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:03.315428 17904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:40:03.315593 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "7585b7acbba641868d271945a613d6b5"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "2908bea76e60f1fe7481203eb8a51b31"
server_key_iv: "cbbe812302af5e59e9f4ce80026f023f"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.315845 17904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:03.329648 17904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:03.333750 17904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:03.334017 17904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:03.352047 17904 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.125:32823
I20260926 07:40:03.352054 17970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.125:32823 every 8 connection(s)
I20260926 07:40:03.353431 17904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
I20260926 07:40:03.356937 17971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:03.358174 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 17904
I20260926 07:40:03.365351 17971 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.125@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:03.381615 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.360172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.11.9.125:32823)
0926 07:40:03.360901 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.360916 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.360954 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:40:03.362870 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.362890 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.363253 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.363672 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.365386 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.366862 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.368377 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.369343 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.369536 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.374142 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.374154 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.374168 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.374206 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.377333 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.377997 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.378005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.378014 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.378182 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.378607 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.378612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.378616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.379296 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.379579 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.379934 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.380316 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":352,"threads_started":1}
I20260926 07:40:03.383492 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.367832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.11.9.126:36123)
0926 07:40:03.368016 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.368023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.368546 (+ 523us) server_negotiation.cc:407] Connection header received
0926 07:40:03.368803 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.368850 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.368935 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.369051 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.370510 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.371592 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.373344 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.373747 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.373901 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.377552 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.377556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.377559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.377601 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.380861 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.381593 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.381601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.381607 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.381759 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.382160 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.382165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.382169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.382456 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.382606 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.383135 (+ 529us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.383280 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:40:03.383728 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.367767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:33712)
0926 07:40:03.368423 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.368466 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.368656 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.369103 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.369111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.370034 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.370303 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.370312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.371755 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.371760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.373170 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.373216 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.373984 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.373992 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.374162 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.374825 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.374851 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.377380 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.380981 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.380989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.380994 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.381459 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.381893 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.381898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.381902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.382041 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.382615 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.382625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.382815 (+ 190us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.383029 (+ 214us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.383331 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":300,"threads_started":1}
I20260926 07:40:03.385238 17971 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.125@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
I20260926 07:40:03.396446 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.385912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40734)
0926 07:40:03.386056 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.386070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.386141 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.386496 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.386500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.386815 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.386925 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.386933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.387829 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.387834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.388681 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.388691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.389397 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.389402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.389567 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.390005 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.390019 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.391846 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.394618 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.394622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.394624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.394952 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.395235 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.395238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.395240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.395350 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.395793 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.395796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.395901 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.396013 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.396192 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 07:40:03.396517 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.386146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.11.9.125:32823)
0926 07:40:03.386290 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.386302 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.386314 (+ 12us) server_negotiation.cc:407] Connection header received
0926 07:40:03.386344 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.386347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.386389 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.386484 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.387046 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.387688 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.388818 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.389268 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.389393 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.392022 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.392026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.392029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.392055 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.394510 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.395044 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.395048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.395050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.395148 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.395436 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.395439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.395441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.395690 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.395789 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.396093 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.396289 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":54}
I20260926 07:40:03.400549 17766 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
I20260926 07:40:03.400974 17971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Bootstrap starting.
I20260926 07:40:03.404285 17971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:03.405368 17971 log.cc:824] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:03.407791 17971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: No bootstrap required, opened a new log
I20260926 07:40:03.414785 17971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } }
I20260926 07:40:03.415118 17971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:03.415221 17971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7585b7acbba641868d271945a613d6b5, State: Initialized, Role: FOLLOWER
I20260926 07:40:03.415804 17971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } }
I20260926 07:40:03.416680 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } }
I20260926 07:40:03.416958 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:03.417840 17971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:40:03.426222 17988 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:40:03.426370 17988 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:40:03.428771 17904 master_runner.cc:428] Detected that this master 7585b7acbba641868d271945a613d6b5 is joining an existing cluster
I20260926 07:40:03.428900 17904 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.9.125:32823
I20260926 07:40:03.448573 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.433006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40740)
0926 07:40:03.434065 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.434081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.434235 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.434759 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.434762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.435526 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.435892 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.435903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.436913 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.436918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.438134 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.438147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.439625 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.439632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.439796 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.440424 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.440446 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.441611 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.445563 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.445568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.445574 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.446109 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.446573 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.446586 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.446754 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.447512 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.447517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.447652 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.447777 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.448102 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":23,"thread_start_us":759,"threads_started":1}
I20260926 07:40:03.449149 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.432508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.11.9.126:36123)
0926 07:40:03.432937 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.432943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.433834 (+ 891us) server_negotiation.cc:407] Connection header received
0926 07:40:03.434099 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.434104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.434221 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.434496 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.435887 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.437014 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.439287 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.439959 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.440057 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.442495 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.442501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.442506 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.442547 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.445873 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446637 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.446642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.446646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.446791 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.447189 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.447194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.447197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.447563 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.447681 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.448258 (+ 577us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.448462 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300}
I20260926 07:40:03.449570 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.432647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.11.9.125:32823)
0926 07:40:03.432876 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.432887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.434158 (+ 1271us) server_negotiation.cc:407] Connection header received
0926 07:40:03.434442 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.434450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.434540 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.434628 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.436060 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.436756 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.438347 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.439145 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.439267 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.441773 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.441783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.441787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.441818 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.445420 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446283 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.446300 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.446305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.446450 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446876 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.446880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.446884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.447313 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.447464 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.448973 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.449208 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":52}
I20260926 07:40:03.450940 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.432553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:33716)
0926 07:40:03.433695 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.433721 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.433943 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.434522 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.434528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.435269 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.435689 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.435702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.437248 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.437255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.439145 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.439155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.440341 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.440348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.440538 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.441127 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.441148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.442284 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.446019 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.446029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.446504 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.446923 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.446932 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.446937 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.447071 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.447764 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.447770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.447945 (+ 175us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.449602 (+ 1657us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.450500 (+ 898us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":80,"thread_start_us":802,"threads_started":1}
WARNING: no policy specified for oryx/127.11.9.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.124@KRBTEST.COM" created.
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408401, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.126@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
I20260926 07:40:03.473601 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.456592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.11.9.125:32823)
0926 07:40:03.456781 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.456790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.460256 (+ 3466us) server_negotiation.cc:407] Connection header received
0926 07:40:03.460528 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.460535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.460649 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.460765 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.461862 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.462769 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.464002 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.464466 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.464603 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.467639 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.467645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.467648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.467674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.470951 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.471613 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.471621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.471624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.471749 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.472153 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.472158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.472161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.472413 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.472567 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.472988 (+ 421us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.473225 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:40:03.473659 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.456623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40746)
0926 07:40:03.460103 (+ 3480us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.460141 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.460312 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.460829 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.460835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.461497 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.461690 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.461699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.462918 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.462926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.463858 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.463869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.464620 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.464625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.464808 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.465456 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.465474 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.467461 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.471078 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.471089 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.471097 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.471481 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.471885 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.471889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.471893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.472034 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.472540 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.472548 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.472716 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.472898 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.473225 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3258,"thread_start_us":3114,"threads_started":1}
I20260926 07:40:03.475613 17442 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.9.125:32823: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7585b7acbba641868d271945a613d6b5"
member_type: NON_VOTER
last_known_addr {
host: "127.11.9.125"
port: 32823
}
attrs {
promote: true
}
}
dest_uuid: "43e5257e3e904e598200dcaf9445dba3"
cas_config_opid_index: -1
I20260926 07:40:03.476555 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } }
I20260926 07:40:03.484822 17946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:03.485073 18001 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7585b7acbba641868d271945a613d6b5 (127.11.9.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.489483 17946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 07:40:03.490513 18000 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.124.keytab.
Entry for principal oryx/127.11.9.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.124.keytab.
I20260926 07:40:03.493626 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:33919
--webserver_interface=127.11.9.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.124.keytab
--principal=oryx/127.11.9.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:40:03.508654 18001 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.9.125:32823
I20260926 07:40:03.509348 18000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.509543 17904 master.cc:537] Master@127.11.9.125:32823 shutting down...
I20260926 07:40:03.509631 18000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:03.525426 17946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7585b7acbba641868d271945a613d6b5 (127.11.9.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.537575 18011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.538105 18011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:03.538528 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e5257e3e904e598200dcaf9445dba3. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.538781 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:03.538597 18006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.538902 18006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:03.563189 17904 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Raft consensus shutting down.
I20260926 07:40:03.563658 17904 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Raft consensus is shut down!
I20260926 07:40:03.563840 17904 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: stopping tablet replica
I20260926 07:40:03.571205 17904 master.cc:552] Master@127.11.9.125:32823 shutdown complete.
I20260926 07:40:03.571323 17904 master_runner.cc:305] Clearing existing system tablet
I20260926 07:40:03.571672 17904 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:03.571781 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:03.573997 17904 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 07:40:03.575425 18013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:03.575721 17904 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:03.576009 17904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 07:40:03.576103 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "7585b7acbba641868d271945a613d6b5"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "2908bea76e60f1fe7481203eb8a51b31"
server_key_iv: "cbbe812302af5e59e9f4ce80026f023f"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.579073 17904 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:03.583539 17904 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 07:40:03.583724 17904 log.cc:1196] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 07:40:03.584193 17904 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Deleting consensus metadata
I20260926 07:40:03.584443 17904 master_runner.cc:315] Copying system tablet from 127.11.9.126:36123
I20260926 07:40:03.588420 17904 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: tablet copy: Beginning tablet copy session from remote peer at address 127.11.9.126:36123
I20260926 07:40:03.605248 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.589140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.11.9.126:36123)
0926 07:40:03.589413 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.589422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.593832 (+ 4410us) server_negotiation.cc:407] Connection header received
0926 07:40:03.594038 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.594044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.594138 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.594242 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:03.595046 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.595769 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.596995 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.597476 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.597626 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.599641 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:03.599647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:03.599658 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:03.599689 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:03.602715 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.603330 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.603337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.603341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.603466 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:03.603849 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:03.603857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:03.603860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:03.604131 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:03.604285 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.604658 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.604873 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 07:40:03.605353 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.589073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:33724)
0926 07:40:03.593703 (+ 4630us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.593723 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.593907 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.594276 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.594280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.594687 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:03.594898 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.594907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.595908 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.595913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.596850 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.596861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.597655 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.597660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.597835 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.598454 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:03.598471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:03.599464 (+ 993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:03.602848 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.602854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.602859 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.603209 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.603574 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:03.603582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:03.603586 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:03.603737 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:03.604245 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:03.604250 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:03.604422 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.604570 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.604873 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4500,"thread_start_us":4369,"threads_started":1}
I20260926 07:40:03.606374 17462 tablet_copy_service.cc:140] P 43e5257e3e904e598200dcaf9445dba3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7585b7acbba641868d271945a613d6b5 ({username='oryx', principal='oryx/127.11.9.125@KRBTEST.COM'} at 127.0.0.1:33724)
I20260926 07:40:03.606619 17462 tablet_copy_service.cc:161] P 43e5257e3e904e598200dcaf9445dba3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7585b7acbba641868d271945a613d6b5 at {username='oryx', principal='oryx/127.11.9.125@KRBTEST.COM'} at 127.0.0.1:33724: session id = 7585b7acbba641868d271945a613d6b5-00000000000000000000000000000000
I20260926 07:40:03.609499 17462 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 07:40:03.611323 17904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 07:40:03.616146 17904 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: tablet copy: Starting download of 0 data blocks...
I20260926 07:40:03.616454 17904 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: tablet copy: Starting download of 1 WAL segments...
I20260926 07:40:03.619329 17904 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 07:40:03.622925 17462 tablet_copy_service.cc:342] P 43e5257e3e904e598200dcaf9445dba3: Request end of tablet copy session 7585b7acbba641868d271945a613d6b5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.9.125@KRBTEST.COM'} at 127.0.0.1:33724
I20260926 07:40:03.623103 17462 tablet_copy_service.cc:433] P 43e5257e3e904e598200dcaf9445dba3: ending tablet copy session 7585b7acbba641868d271945a613d6b5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7585b7acbba641868d271945a613d6b5
I20260926 07:40:03.634248 17904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:03.638860 18020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:03.640225 18021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:03.644356 18023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:03.646466 17904 server_base.cc:1034] running on GCE node
I20260926 07:40:03.646818 17904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:03.649391 17904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:03.650617 17904 hybrid_clock.cc:648] HybridClock initialized: now 1790408403650533 us; error 59 us; skew 500 ppm
I20260926 07:40:03.650826 17904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:03.652967 17904 init.cc:377] Logged in from keytab as oryx/127.11.9.125@KRBTEST.COM (short username oryx)
I20260926 07:40:03.658115 17904 webserver.cc:460] Webserver started at http://127.11.9.125:37923/ using document root <none> and password file <none>
I20260926 07:40:03.658476 17904 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:03.658560 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:03.661590 17904 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 07:40:03.663573 18030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:03.663988 17904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 07:40:03.664108 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "7585b7acbba641868d271945a613d6b5"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "2908bea76e60f1fe7481203eb8a51b31"
server_key_iv: "cbbe812302af5e59e9f4ce80026f023f"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.664268 17904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:03.677091 17904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:03.681106 17904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:03.715590 17904 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.125:32823
I20260926 07:40:03.715682 18081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.125:32823 every 8 connection(s)
I20260926 07:40:03.716737 17904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
I20260926 07:40:03.721998 18082 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:40:03.725977 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Bootstrap starting.
I20260926 07:40:03.740433 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:40:03.741065 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: Bootstrap complete.
I20260926 07:40:03.742476 18082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } }
I20260926 07:40:03.742810 18082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7585b7acbba641868d271945a613d6b5, State: Initialized, Role: LEARNER
I20260926 07:40:03.743152 18082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } }
I20260926 07:40:03.743986 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.744349 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:03.744403 18082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:03.752615 18096 catalog_manager.cc:1259] Loaded cluster ID: 65811b5f4021461f985a334f6a402d4e
I20260926 07:40:03.752725 18096 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: loading cluster ID for follower catalog manager: success
I20260926 07:40:03.755095 18096 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: acquiring CA information for follower catalog manager: success
I20260926 07:40:03.756825 18096 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:40:03.886780 17899 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
W20260926 07:40:03.904184 18003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:03.904516 18003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:03.904668 18003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:03.912459 18003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:03.912600 18003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:03.912704 18003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:03.912793 18003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:03.912858 18003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:03.923233 18003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.124.keytab
--principal=oryx/127.11.9.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:33919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.9.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:03.924572 18003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:03.926003 18003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 07:40:03.931967 18001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.932245 18001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:03.941449 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.932661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40748)
0926 07:40:03.932984 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:03.933008 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:03.933190 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:03.934113 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:03.934119 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:03.934147 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:03.934344 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.934355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.936851 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.936857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.938977 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:03.938994 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.940671 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.940677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:03.940886 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.940941 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 07:40:03.941011 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 07:40:03.941179 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
W20260926 07:40:03.941529 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:03.942016 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:03.942723 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:03.932906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.11.9.125:32823)
0926 07:40:03.933734 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:03.933741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:03.933761 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:40:03.933841 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:03.933851 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:03.933930 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:03.934489 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:03.934690 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.936696 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.939274 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:03.940490 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:03.941816 (+ 1326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:03.942071 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:03.942140 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 07:40:03.942292 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":552,"threads_started":1}
I20260926 07:40:03.943714 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e5257e3e904e598200dcaf9445dba3. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:03.944025 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:03.944648 18000 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: attempting to promote NON_VOTER 7585b7acbba641868d271945a613d6b5 to VOTER
W20260926 07:40:03.945021 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:03.945798 18000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } }
I20260926 07:40:03.945935 18003 server_base.cc:1034] running on GCE node
I20260926 07:40:03.946884 18003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:03.947751 18057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 LEARNER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 07:40:03.948398 18002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 07:40:03.949311 18003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:03.950532 18003 hybrid_clock.cc:648] HybridClock initialized: now 1790408403950460 us; error 62 us; skew 500 ppm
I20260926 07:40:03.950942 18003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:03.951498 18000 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 7585b7acbba641868d271945a613d6b5 (127.11.9.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } }
I20260926 07:40:03.952761 18057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 7585b7acbba641868d271945a613d6b5 (127.11.9.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } }
I20260926 07:40:03.955243 18001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } }
I20260926 07:40:03.955467 18001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:03.955788 18000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } }
I20260926 07:40:03.955973 18000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:03.956909 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } }
I20260926 07:40:03.957129 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 07:40:03 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:03.961545 18003 init.cc:377] Logged in from keytab as oryx/127.11.9.124@KRBTEST.COM (short username oryx)
I20260926 07:40:03.963953 18003 webserver.cc:460] Webserver started at http://127.11.9.124:46565/ using document root <none> and password file <none>
I20260926 07:40:03.964838 18003 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:03.964975 18003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:03.965392 18003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:03.969445 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/instance:
uuid: "15db82b6b33f4accaa139c11b2f3ae83"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "370dde30ae3934aba8f306de59fd2dec"
server_key_iv: "d8005adac75afb7a469e8bbeed12bd5e"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.970378 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal/instance:
uuid: "15db82b6b33f4accaa139c11b2f3ae83"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "370dde30ae3934aba8f306de59fd2dec"
server_key_iv: "d8005adac75afb7a469e8bbeed12bd5e"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.977020 18003 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260926 07:40:03.981602 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:03.983197 18003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:40:03.983381 18003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "15db82b6b33f4accaa139c11b2f3ae83"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "370dde30ae3934aba8f306de59fd2dec"
server_key_iv: "d8005adac75afb7a469e8bbeed12bd5e"
server_key_version: "encryptionkey@0"
I20260926 07:40:03.983601 18003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:04.014724 18003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:04.019322 18003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:04.019611 18003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:04.037879 18003 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.124:33919
I20260926 07:40:04.037909 18164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.124:33919 every 8 connection(s)
I20260926 07:40:04.039242 18003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
I20260926 07:40:04.042904 18165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:04.048174 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18003
I20260926 07:40:04.050343 18165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.124@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
I20260926 07:40:04.068068 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.052836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.11.9.126:36123)
0926 07:40:04.053043 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.053048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.053484 (+ 436us) server_negotiation.cc:407] Connection header received
0926 07:40:04.053837 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.053842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.053924 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.054029 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.055225 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.055982 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.057476 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.057929 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.058063 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.061361 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.061366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.061369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.061398 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.064696 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.065615 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.065619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.065621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.065737 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.066203 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.066208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.066211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.066562 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.066713 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.067502 (+ 789us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.067694 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:40:04.069200 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.050283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.11.9.124:33919)
0926 07:40:04.051052 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.051064 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.051127 (+ 63us) server_negotiation.cc:407] Connection header received
0926 07:40:04.053165 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.053224 (+ 59us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.053658 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.053887 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.055273 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.056421 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.057727 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.058238 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.058490 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.062563 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.062573 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.062587 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.062622 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.065932 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.066695 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.066703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.066706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.066853 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.067189 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.067193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.067196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.067582 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.067848 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.068160 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.068395 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":574,"thread_start_us":369,"threads_started":1}
I20260926 07:40:04.069242 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.052687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:33732)
0926 07:40:04.053353 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.053392 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.053621 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.054095 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.054101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.054700 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.055069 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.055084 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.056117 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.056122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.057349 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.057359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.058143 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.058149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.058353 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.059106 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.059126 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.061170 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.064826 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.064841 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.064853 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.065500 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.065844 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.065851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.065857 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.066089 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.066695 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.066703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.067085 (+ 382us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.067409 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.067880 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":34,"thread_start_us":279,"threads_started":1}
I20260926 07:40:04.071431 18165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.124@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
I20260926 07:40:04.083328 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.072112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40760)
0926 07:40:04.072284 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.072302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.072390 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.072640 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.072643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.072932 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.073046 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.073053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.074056 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.074059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.074916 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.074926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.075556 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.075560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.075687 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.076267 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.076288 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.078533 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.081551 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.081554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.081557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.081894 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.082196 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.082199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.082201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.082310 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.082716 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.082719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.082820 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.082930 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.083117 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 07:40:04.083365 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.072237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.11.9.125:32823)
0926 07:40:04.072410 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.072414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.072430 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:40:04.072476 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.072479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.072533 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.072610 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.073212 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.073922 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.075033 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.075427 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.075525 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.078716 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.078720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.078724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.078755 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.081446 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.081987 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.081991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.081993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.082110 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.082396 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.082399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.082401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.082627 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.082727 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.083005 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.083154 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:40:04.084271 18165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408403, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.124@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
I20260926 07:40:04.095376 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.084996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.0.0.1:39018)
0926 07:40:04.085114 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.085132 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.085293 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.085535 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.085538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.085823 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.085952 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.085959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.086818 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.086821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.087783 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.087795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.088481 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.088485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.088636 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.089094 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.089109 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.090951 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.093466 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.093470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.093472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.093805 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.094111 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.094113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.094116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.094245 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.094655 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.094658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.094755 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.094907 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.095069 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 07:40:04.095403 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.084939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.11.9.124:33919)
0926 07:40:04.085094 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.085098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.085224 (+ 126us) server_negotiation.cc:407] Connection header received
0926 07:40:04.085380 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.085383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.085430 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.085501 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.086060 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.086686 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.087917 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.088357 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.088459 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.091110 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.091115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.091118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.091144 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.093363 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.093906 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.093909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.093911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.094023 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.094331 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.094334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.094336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.094564 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.094665 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.094986 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.095168 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":63}
I20260926 07:40:04.100104 18165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Bootstrap starting.
I20260926 07:40:04.103281 18165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:04.104254 18165 log.cc:824] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:04.106750 18165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: No bootstrap required, opened a new log
I20260926 07:40:04.113605 18165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } }
I20260926 07:40:04.113914 18165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:04.114024 18165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15db82b6b33f4accaa139c11b2f3ae83, State: Initialized, Role: FOLLOWER
I20260926 07:40:04.114545 18165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } }
I20260926 07:40:04.115327 18171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } } }
I20260926 07:40:04.115650 18171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:04.116366 18165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:04.123682 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17504
W20260926 07:40:04.125201 18182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:40:04.125335 18182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:40:04.129675 18003 master_runner.cc:428] Detected that this master 15db82b6b33f4accaa139c11b2f3ae83 is joining an existing cluster
I20260926 07:40:04.129796 18003 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.9.124:33919
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.65@KRBTEST.COM
I20260926 07:40:04.142918 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.125403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.11.9.65:43457)
0926 07:40:04.126061 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.126067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.126090 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:40:04.126147 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.126152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.126375 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.126540 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.128091 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.129470 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.130998 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.131554 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.131711 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.136731 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.136741 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.136754 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.136784 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.139918 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.140562 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.140571 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.140574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.140706 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.141073 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.141079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.141083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.141569 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.141858 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.142183 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.142455 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":338,"threads_started":1}
I20260926 07:40:04.144354 17627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:04.196223 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.144987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.11.9.125:32823)
0926 07:40:04.145417 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.145426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.162130 (+ 16704us) server_negotiation.cc:407] Connection header received
0926 07:40:04.163575 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.163585 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.163699 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.163784 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.168053 (+ 4269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.169512 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.177256 (+ 7744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.178234 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.178379 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.187417 (+ 9038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.187426 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.187433 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.187477 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.190904 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.193693 (+ 2789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.193701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.193707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.193863 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.194389 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.194396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.194401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.194745 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.194889 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.195538 (+ 649us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.195796 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260926 07:40:04.198310 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.143686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:40772)
0926 07:40:04.161930 (+ 18244us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.161969 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.162691 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.164101 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.164108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.165079 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.165884 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.165902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.170238 (+ 4336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.170246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.172559 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.172583 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.179279 (+ 6696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.179288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.179547 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.182805 (+ 3258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.182843 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.184368 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.191074 (+ 6706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.191079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.191083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.191570 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.194024 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.194031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.194040 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.194235 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.194976 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.194982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.195188 (+ 206us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.197081 (+ 1893us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.197447 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17995,"thread_start_us":43217,"threads_started":1}
I20260926 07:40:04.200584 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.145583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.11.9.126:36123)
0926 07:40:04.145794 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.145803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.163433 (+ 17630us) server_negotiation.cc:407] Connection header received
0926 07:40:04.166215 (+ 2782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.166223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.166321 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.166412 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.173034 (+ 6622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.174534 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.180291 (+ 5757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.181232 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.181369 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.191776 (+ 10407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.191783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.191787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.191827 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.195422 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.197078 (+ 1656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.197083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.197087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.197267 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.197723 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.197728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.197731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.198072 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.198195 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.199946 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.200174 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:40:04.202270 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.144490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:33748)
0926 07:40:04.163295 (+ 18805us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.163318 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.166057 (+ 2739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.166630 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.166636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.167447 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.167842 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.167856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.174767 (+ 6911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.174776 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.177023 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.177041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.181715 (+ 4674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.181724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.181970 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.184837 (+ 2867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.184866 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.186411 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.196513 (+ 10102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.196518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.196521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.196934 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.197407 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.197411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.197415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.197572 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.198269 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.198276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.198563 (+ 287us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.201383 (+ 2820us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.201773 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18636,"spinlock_wait_cycles":1693952,"thread_start_us":17670,"threads_started":1}
I20260926 07:40:04.205965 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.146214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.11.9.124:33919)
0926 07:40:04.146429 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.146437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.187108 (+ 40671us) server_negotiation.cc:407] Connection header received
0926 07:40:04.187328 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.187336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.187399 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.187476 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.188865 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.190043 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.191967 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.192760 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.192862 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.198717 (+ 5855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.198723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.198728 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.198759 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.202273 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.203959 (+ 1686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.203964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.203968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.204100 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.204425 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.204430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.204433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.204784 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.204904 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.205421 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.205611 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:40:04.207231 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.146307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.0.0.1:39022)
0926 07:40:04.187013 (+ 40706us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.187032 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.187184 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.187637 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.187641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.188347 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.188707 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.188719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.190251 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.190255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.191799 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.191811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.193181 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.193187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.193362 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.195777 (+ 2415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.195798 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.198524 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.202398 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.202403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.202407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.203512 (+ 1105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.204209 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.204214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.204218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.204332 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.204991 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.204996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.205183 (+ 187us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.206150 (+ 967us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.206383 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40551,"thread_start_us":26377,"threads_started":1}
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408401, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.126@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
I20260926 07:40:04.223598 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.207478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.0.0.1:39032)
0926 07:40:04.207836 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.207857 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.207959 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.208979 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.208985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.209451 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.209646 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.209657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.210999 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.211005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.212422 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.212436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.213435 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.213441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.213610 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.214250 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.214277 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.216932 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.220986 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.220992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.220996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.221478 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.221863 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.221868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.221872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.222012 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.222621 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.222627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.222789 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.222940 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.223223 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260926 07:40:04.224030 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.207776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.11.9.124:33919)
0926 07:40:04.208622 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.208627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.208642 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:40:04.208720 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.208725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.208828 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.208919 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.209804 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.210866 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.212571 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.213275 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.213391 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.217151 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.217192 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.217197 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.217239 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.220850 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.221599 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.221605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.221609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.221746 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.222122 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.222127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.222131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.222496 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.222640 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.223043 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.223422 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749}
I20260926 07:40:04.225486 17442 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.9.124:33919: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83"
member_type: NON_VOTER
last_known_addr {
host: "127.11.9.124"
port: 33919
}
attrs {
promote: true
}
}
dest_uuid: "43e5257e3e904e598200dcaf9445dba3"
cas_config_opid_index: 6
I20260926 07:40:04.226209 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } }
I20260926 07:40:04.228397 18057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 07:40:04.228894 18140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:04.232681 18140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 07:40:04.233296 18001 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 07:40:04.233866 18001 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:04.235863 18002 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.239699 18002 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.9.124:33919
I20260926 07:40:04.239984 18002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.240272 18002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.240516 18002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.240754 18002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.241573 18003 master.cc:537] Master@127.11.9.124:33919 shutting down...
I20260926 07:40:04.242890 18057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.249663 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.249935 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:04.253278 18139 raft_consensus.cc:858] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } }
W20260926 07:40:04.277501 18119 tablet.cc:2369] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:40:04.253459 18139 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } }
I20260926 07:40:04.284677 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17504
I20260926 07:40:04.290244 18139 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7585b7acbba641868d271945a613d6b5 (127.11.9.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:04.302484 18171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e5257e3e904e598200dcaf9445dba3. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:04.302719 18171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 07:40:04.303067 18195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:04.303375 18195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 07:40:04.305001 18198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: NON_VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: true } } }
I20260926 07:40:04.305222 18198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 07:40:04.305550 18001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.305845 18001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
W20260926 07:40:04.307350 17427 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260926 07:40:04.308394 18003 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260926 07:40:04.308854 18003 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260926 07:40:04.309046 18003 tablet_replica.cc:330] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: stopping tablet replica
I20260926 07:40:04.310077 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:43457
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:36123
--webserver_port=45721
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:04.317664 17427 consensus_peers.cc:487] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 -> Peer 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124:33919): Couldn't send request to peer 15db82b6b33f4accaa139c11b2f3ae83. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:40:04.321144 18003 master.cc:552] Master@127.11.9.124:33919 shutdown complete.
I20260926 07:40:04.321249 18003 master_runner.cc:305] Clearing existing system tablet
I20260926 07:40:04.321640 18003 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:04.321760 18003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:04.324920 18003 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 07:40:04.326817 18202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:04.327230 18003 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:04.327692 18003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 07:40:04.327853 18003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "15db82b6b33f4accaa139c11b2f3ae83"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "370dde30ae3934aba8f306de59fd2dec"
server_key_iv: "d8005adac75afb7a469e8bbeed12bd5e"
server_key_version: "encryptionkey@0"
I20260926 07:40:04.333608 18003 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:04.341354 18003 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 07:40:04.341578 18003 log.cc:1196] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 07:40:04.342111 18003 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Deleting consensus metadata
I20260926 07:40:04.342406 18003 master_runner.cc:315] Copying system tablet from 127.11.9.126:36123
I20260926 07:40:04.345058 18003 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: tablet copy: Beginning tablet copy session from remote peer at address 127.11.9.126:36123
I20260926 07:40:04.360778 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.345974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.11.9.126:36123)
0926 07:40:04.346253 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.346259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.346603 (+ 344us) server_negotiation.cc:407] Connection header received
0926 07:40:04.346868 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.346874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.346959 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.347025 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.348279 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.349474 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.351415 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.352042 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.352137 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.354568 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.354576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.354581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.354616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.357845 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.358592 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.358598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.358601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.358756 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.359209 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.359214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.359217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.359517 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.359641 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.360191 (+ 550us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.360426 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 07:40:04.361974 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.345979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49190)
0926 07:40:04.346499 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.346522 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.346726 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.347250 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.347255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.347854 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.348113 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.348125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.349660 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.349669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.351262 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.351278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.352426 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.352432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.352588 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.353307 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.353328 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.354374 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.357991 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.357999 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.358006 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.358463 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.358903 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.358907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.358911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.359077 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.359747 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.359752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.359920 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.361248 (+ 1328us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.361553 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":241,"threads_started":1}
I20260926 07:40:04.362463 17462 tablet_copy_service.cc:140] P 43e5257e3e904e598200dcaf9445dba3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 15db82b6b33f4accaa139c11b2f3ae83 ({username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:49190)
I20260926 07:40:04.362622 17462 tablet_copy_service.cc:161] P 43e5257e3e904e598200dcaf9445dba3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 15db82b6b33f4accaa139c11b2f3ae83 at {username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:49190: session id = 15db82b6b33f4accaa139c11b2f3ae83-00000000000000000000000000000000
I20260926 07:40:04.366039 17462 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 07:40:04.367702 18003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 07:40:04.374441 18003 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: tablet copy: Starting download of 0 data blocks...
I20260926 07:40:04.374737 18003 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: tablet copy: Starting download of 1 WAL segments...
I20260926 07:40:04.377099 18003 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 07:40:04.382082 17462 tablet_copy_service.cc:342] P 43e5257e3e904e598200dcaf9445dba3: Request end of tablet copy session 15db82b6b33f4accaa139c11b2f3ae83-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:49190
I20260926 07:40:04.382225 17462 tablet_copy_service.cc:433] P 43e5257e3e904e598200dcaf9445dba3: ending tablet copy session 15db82b6b33f4accaa139c11b2f3ae83-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 15db82b6b33f4accaa139c11b2f3ae83
I20260926 07:40:04.393723 18003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:04.396242 18209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:04.397426 18210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:04.401607 18003 server_base.cc:1034] running on GCE node
W20260926 07:40:04.402064 18212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:04.402500 18003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:04.404390 18003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:04.405572 18003 hybrid_clock.cc:648] HybridClock initialized: now 1790408404405540 us; error 23 us; skew 500 ppm
I20260926 07:40:04.405838 18003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408404, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:04.408041 18003 init.cc:377] Logged in from keytab as oryx/127.11.9.124@KRBTEST.COM (short username oryx)
I20260926 07:40:04.411546 18003 webserver.cc:460] Webserver started at http://127.11.9.124:37399/ using document root <none> and password file <none>
I20260926 07:40:04.411902 18003 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:04.412007 18003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:04.415234 18003 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 07:40:04.417026 18219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:04.417454 18003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260926 07:40:04.417573 18003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "15db82b6b33f4accaa139c11b2f3ae83"
format_stamp: "Formatted at 2026-09-26 07:40:03 on dist-test-slave-x492"
server_key: "370dde30ae3934aba8f306de59fd2dec"
server_key_iv: "d8005adac75afb7a469e8bbeed12bd5e"
server_key_version: "encryptionkey@0"
I20260926 07:40:04.417771 18003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:04.441057 18003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:04.445530 18003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:04.476645 18003 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.124:33919
I20260926 07:40:04.476703 18270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.124:33919 every 8 connection(s)
I20260926 07:40:04.477727 18003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
I20260926 07:40:04.483543 18271 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:40:04.487450 18271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Bootstrap starting.
I20260926 07:40:04.503579 18271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:40:04.504133 18271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Bootstrap complete.
I20260926 07:40:04.505285 18271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } }
I20260926 07:40:04.505652 18271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 15db82b6b33f4accaa139c11b2f3ae83, State: Initialized, Role: LEARNER
I20260926 07:40:04.506187 18271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } }
I20260926 07:40:04.507136 18274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.507498 18271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:04.507609 18274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:04.516016 18285 catalog_manager.cc:1259] Loaded cluster ID: 65811b5f4021461f985a334f6a402d4e
I20260926 07:40:04.516211 18285 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: loading cluster ID for follower catalog manager: success
I20260926 07:40:04.519959 18285 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: acquiring CA information for follower catalog manager: success
I20260926 07:40:04.522411 18285 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 07:40:04.685710 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:04.686007 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:04.686172 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:04.693480 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:04.693607 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:04.693763 18200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:04.703717 18200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:43457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=45721
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:04.704897 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:04.706192 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:04.720566 18291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:04.720816 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:04.721570 18290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:04.721951 18200 server_base.cc:1034] running on GCE node
I20260926 07:40:04.722752 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:04.723994 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:04.725266 18200 hybrid_clock.cc:648] HybridClock initialized: now 1790408404725184 us; error 59 us; skew 500 ppm
I20260926 07:40:04.725603 18200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408404, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:04.729065 18200 init.cc:377] Logged in from keytab as oryx/127.11.9.65@KRBTEST.COM (short username oryx)
I20260926 07:40:04.734318 18200 webserver.cc:460] Webserver started at http://127.11.9.65:45721/ using document root <none> and password file <none>
I20260926 07:40:04.735021 18200 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:04.735196 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:04.740756 18200 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:04.743486 18300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:04.744840 18200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:40:04.745024 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "9d656517c6e145fb8bfeacddd154fd03"
format_stamp: "Formatted at 2026-09-26 07:40:01 on dist-test-slave-x492"
server_key: "52e01e517d16a41d472b063042895e56"
server_key_iv: "48a26828b68bc429593d46a2c795f81e"
server_key_version: "encryptionkey@0"
I20260926 07:40:04.746238 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:04.764221 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:04.768126 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:04.768404 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:04.769331 18200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:04.770598 18200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:04.770715 18200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:04.770841 18200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:04.770936 18200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:04.806442 18200 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:43457
I20260926 07:40:04.806491 18412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:43457 every 8 connection(s)
I20260926 07:40:04.807943 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:04.809896 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18200
I20260926 07:40:04.810253 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17638
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.66@KRBTEST.COM
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408404, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408404, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
Sep 26 07:40:04 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408404, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
I20260926 07:40:04.843844 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.812671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:43541 (local address 127.11.9.125:32823)
0926 07:40:04.814765 (+ 2094us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.814774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.814809 (+ 35us) server_negotiation.cc:407] Connection header received
0926 07:40:04.823169 (+ 8360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.823175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.823265 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.823344 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.826089 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827195 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.830136 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.830781 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.830875 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.836609 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.836614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.836623 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.836652 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.840212 (+ 3560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.841154 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.841186 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.841190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.841322 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.841834 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.841842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.841847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.842216 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.842336 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.843307 (+ 971us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.843516 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1939,"thread_start_us":1788,"threads_started":1}
I20260926 07:40:04.844303 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.811054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:46345 (local address 127.11.9.126:36123)
0926 07:40:04.811331 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.811338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.821094 (+ 9756us) server_negotiation.cc:407] Connection header received
0926 07:40:04.823162 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.823168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.823253 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.823331 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.826089 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827177 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.830135 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.830787 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.830875 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.836352 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.836358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.836362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.836402 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.840101 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842061 (+ 1960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.842066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.842070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.842211 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842698 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.842703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.842707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.843049 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.843179 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.843731 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.843935 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260926 07:40:04.845556 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.811818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.11.9.65:43541)
0926 07:40:04.813966 (+ 2148us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.814025 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.822833 (+ 8808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.823798 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.823810 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.824660 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.825903 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.825916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827376 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.827381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.829964 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.829978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.831215 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.831222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.831573 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.832474 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.832518 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.836418 (+ 3900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.840345 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.840367 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.840389 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.840971 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.841436 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.841440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.841444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.841696 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.842443 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.842453 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.842880 (+ 427us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.844132 (+ 1252us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.844359 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1776,"mutex_wait_us":561,"spinlock_wait_cycles":50560,"thread_start_us":1434,"threads_started":1}
I20260926 07:40:04.846194 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.815469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:33019 (local address 127.11.9.124:33919)
0926 07:40:04.822219 (+ 6750us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.822226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.822244 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:40:04.823286 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.823292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.823371 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.823464 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.826096 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827483 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.830530 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.831205 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.831344 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.838109 (+ 6765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.838116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.838120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.838158 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.841277 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842207 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.842212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.842215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.842350 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842803 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.842808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.842811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.843126 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.843260 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.845642 (+ 2382us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.845845 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6618,"thread_start_us":6503,"threads_started":1}
I20260926 07:40:04.846421 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.815479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.11.9.65:33019)
0926 07:40:04.821331 (+ 5852us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.821351 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.822833 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.823798 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.823810 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.825198 (+ 1388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.825892 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.825916 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827655 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.827660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.830350 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.830365 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.831625 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.831632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.831874 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.832647 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.832673 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.837868 (+ 5195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.841427 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.841432 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.841436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.842072 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.842476 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.842483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.842677 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.843313 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.843318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.843437 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.843605 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.844107 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5704,"mutex_wait_us":669,"thread_start_us":5164,"threads_started":1}
I20260926 07:40:04.846714 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.811284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.11.9.66:43065)
0926 07:40:04.812486 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.812492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.822195 (+ 9703us) server_negotiation.cc:407] Connection header received
0926 07:40:04.822291 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.822299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.822496 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.822682 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:04.824937 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.826558 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.828409 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.829226 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.829340 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.837318 (+ 7978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:04.837332 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:04.837352 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:04.837398 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:04.842223 (+ 4825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.843066 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.843075 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.843081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.843223 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:04.843632 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:04.843636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:04.843639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:04.844267 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:04.845757 (+ 1490us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.845908 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.846271 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":918,"threads_started":1}
I20260926 07:40:04.847299 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.812969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:46345)
0926 07:40:04.820969 (+ 8000us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.820991 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.822963 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.823801 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.823812 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.824660 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:04.825899 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.825916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.827376 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.827381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.829964 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.829978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.831215 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.831222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.831574 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.832474 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:04.832518 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:04.836068 (+ 3550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:04.841254 (+ 5186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.841259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.841263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.841915 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.842373 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:04.842378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:04.842381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:04.842564 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:04.843257 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:04.843264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:04.843434 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.844539 (+ 1105us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.844803 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7851,"mutex_wait_us":503,"spinlock_wait_cycles":13696,"thread_start_us":8040,"threads_started":1}
I20260926 07:40:04.847600 18415 heartbeater.cc:347] Connected to a master server at 127.11.9.125:32823
I20260926 07:40:04.847640 18413 heartbeater.cc:347] Connected to a master server at 127.11.9.124:33919
I20260926 07:40:04.847972 18415 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:04.847972 18413 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:04.848170 17761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:04.848865 18415 heartbeater.cc:511] Master 127.11.9.125:32823 requested a full tablet report, sending...
I20260926 07:40:04.849040 18414 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:04.849105 18413 heartbeater.cc:511] Master 127.11.9.124:33919 requested a full tablet report, sending...
I20260926 07:40:04.849215 18414 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:04.849736 18414 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:04.850889 17442 ts_manager.cc:194] Re-registered known tserver with Master: 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457)
I20260926 07:40:04.851569 18047 ts_manager.cc:194] Registered new tserver with Master: 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457)
I20260926 07:40:04.851686 18236 ts_manager.cc:194] Registered new tserver with Master: 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457)
I20260926 07:40:04.852025 17442 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:46345
I20260926 07:40:04.895272 18425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.895687 18425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.911154 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.896215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.11.9.124:33919)
0926 07:40:04.899538 (+ 3323us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:04.899547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:04.906240 (+ 6693us) server_negotiation.cc:407] Connection header received
0926 07:40:04.906467 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:04.906471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:04.906554 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:04.906683 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:04.907082 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.907776 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.909389 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.910248 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.910406 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.910651 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:04.910756 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 07:40:04.910879 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3156}
I20260926 07:40:04.911198 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:04.895426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.0.0.1:59288)
0926 07:40:04.906127 (+ 10701us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:04.906151 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:04.906309 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:04.906706 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:04.906711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:04.906737 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:04.906928 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.906936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.907921 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.907927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.909247 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:04.909258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:04.910426 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:04.910432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:04.910600 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:04.910635 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 07:40:04.910736 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 07:40:04.910877 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10546,"thread_start_us":10420,"threads_started":1}
I20260926 07:40:04.911924 18246 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260926 07:40:04.912413 18274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e5257e3e904e598200dcaf9445dba3. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: NON_VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: true } } }
I20260926 07:40:04.912751 18274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: LEARNER
I20260926 07:40:04.913017 18425 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: attempting to promote NON_VOTER 15db82b6b33f4accaa139c11b2f3ae83 to VOTER
I20260926 07:40:04.913641 18425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } }
I20260926 07:40:04.915375 18246 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 LEARNER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 07:40:04.915426 18057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Refusing update from remote peer 43e5257e3e904e598200dcaf9445dba3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 07:40:04.915977 18427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 07:40:04.916337 18426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 07:40:04.918926 18426 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.919682 18246 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.922611 18274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.922851 18274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:04.922713 18426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.922982 18426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.923182 18426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.923381 18426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.924490 18057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.935523 18425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.935771 18425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:04.937561 18433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43e5257e3e904e598200dcaf9445dba3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:04.938135 18433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:04.955328 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17638
I20260926 07:40:04.973218 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:43065
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:36123
--webserver_port=35331
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:05.303506 18434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:05.303887 18434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:05.304026 18434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:05.311504 18434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:05.311601 18434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:05.311719 18434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:05.321322 18434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=35331
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:05.322669 18434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:05.323900 18434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:05.336740 18440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:05.336871 18439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:05.336943 18442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:05.338213 18434 server_base.cc:1034] running on GCE node
I20260926 07:40:05.338969 18434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:05.339951 18434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:05.341234 18434 hybrid_clock.cc:648] HybridClock initialized: now 1790408405341122 us; error 52 us; skew 500 ppm
I20260926 07:40:05.341558 18434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:05.344698 18434 init.cc:377] Logged in from keytab as oryx/127.11.9.66@KRBTEST.COM (short username oryx)
I20260926 07:40:05.346091 18434 webserver.cc:460] Webserver started at http://127.11.9.66:35331/ using document root <none> and password file <none>
I20260926 07:40:05.347034 18434 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:05.347159 18434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:05.352718 18434 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:40:05.355244 18449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:05.356520 18434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:40:05.356642 18434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "22756c7c2632488bb848bd512965b705"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e1bab7a5ce7ee0485cbd42a12f5710a1"
server_key_iv: "9767f7bc4e238a8eff4bc27ff354a505"
server_key_version: "encryptionkey@0"
I20260926 07:40:05.357595 18434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:05.375943 18434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:05.379817 18434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:05.380045 18434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:05.380892 18434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:05.382244 18434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:05.382367 18434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:05.382498 18434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:05.382588 18434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:05.416378 18434 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:43065
I20260926 07:40:05.416462 18561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:43065 every 8 connection(s)
I20260926 07:40:05.417943 18434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:05.423218 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18434
I20260926 07:40:05.423525 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17771
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.67@KRBTEST.COM
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:05.472064 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.424633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.11.9.67:36999)
0926 07:40:05.444776 (+ 20143us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:05.444790 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:05.446421 (+ 1631us) server_negotiation.cc:407] Connection header received
0926 07:40:05.446701 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:05.446716 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:05.447168 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:05.447437 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:05.451032 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.452875 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.455043 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.455972 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.456109 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.462641 (+ 6532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:05.462660 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:05.462678 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:05.462737 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:05.467609 (+ 4872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.468395 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.468404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.468408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.468603 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.469104 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.469109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.469113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.469607 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:05.469925 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:05.471271 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0926 07:40:05.471565 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19961,"thread_start_us":19823,"threads_started":1}
I20260926 07:40:05.473738 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.421469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38461 (local address 127.11.9.125:32823)
0926 07:40:05.437215 (+ 15746us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:05.437426 (+ 211us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:05.451582 (+ 14156us) server_negotiation.cc:407] Connection header received
0926 07:40:05.451749 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:05.451758 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:05.451879 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:05.452006 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:05.455100 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.456225 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.459002 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.459714 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.459832 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.466701 (+ 6869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:05.466709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:05.466715 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:05.466759 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:05.470363 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.471271 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.471277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.471282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.471427 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.471907 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.471913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.471916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.472315 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:05.472468 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:05.473072 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 07:40:05.473303 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15544,"thread_start_us":15837,"threads_started":1}
I20260926 07:40:05.473834 17894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:05.474788 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.421503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.11.9.66:38945)
0926 07:40:05.451887 (+ 30384us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:05.451905 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:05.452015 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:05.452567 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:05.452572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:05.453849 (+ 1277us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:05.455115 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.455140 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.456578 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.456583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.458737 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.458749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.459818 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.459825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.460200 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.461104 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:05.461149 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:05.464177 (+ 3028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:05.470024 (+ 5847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.470033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.470052 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.470770 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.471201 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.471205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.471210 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.471432 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.472037 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:05.472045 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:05.472478 (+ 433us) client_negotiation.cc:769] Sending connection context
0926 07:40:05.472873 (+ 395us) client_negotiation.cc:240] Negotiation successful
0926 07:40:05.473411 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30250,"mutex_wait_us":1487,"spinlock_wait_cycles":49536,"thread_start_us":26655,"threads_started":1}
I20260926 07:40:05.475687 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.421977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.11.9.66:38461)
0926 07:40:05.451465 (+ 29488us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:05.451485 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:05.451615 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:05.452541 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:05.452558 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:05.453858 (+ 1300us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:05.454867 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.454898 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.456417 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.456421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.458844 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.458861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.460055 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.460059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.460321 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.461273 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:05.461303 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:05.466449 (+ 5146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:05.470520 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.470525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.470529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.471128 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.471585 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.471592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.471597 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.471773 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.472557 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:05.472562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:05.472753 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 07:40:05.474036 (+ 1283us) client_negotiation.cc:240] Negotiation successful
0926 07:40:05.474443 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29364,"mutex_wait_us":145,"thread_start_us":25977,"threads_started":1}
I20260926 07:40:05.477207 18564 heartbeater.cc:347] Connected to a master server at 127.11.9.125:32823
I20260926 07:40:05.477504 18564 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:05.478216 18564 heartbeater.cc:511] Master 127.11.9.125:32823 requested a full tablet report, sending...
I20260926 07:40:05.479804 18047 ts_manager.cc:194] Registered new tserver with Master: 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:05.482503 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.420810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38945 (local address 127.11.9.124:33919)
0926 07:40:05.447389 (+ 26579us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:05.447399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:05.452115 (+ 4716us) server_negotiation.cc:407] Connection header received
0926 07:40:05.452199 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:05.452205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:05.452289 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:05.452388 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:05.455301 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.456427 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.458893 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.459575 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.459714 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.464436 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:05.464442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:05.464446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:05.464525 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:05.469854 (+ 5329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.470916 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.470923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.470927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.471067 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.471568 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.471573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.471577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.471913 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:05.472060 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:05.475631 (+ 3571us) server_negotiation.cc:299] Negotiation successful
0926 07:40:05.482072 (+ 6441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26403,"thread_start_us":32687,"threads_started":1}
I20260926 07:40:05.483492 18562 heartbeater.cc:347] Connected to a master server at 127.11.9.124:33919
I20260926 07:40:05.483558 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.420765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:34813 (local address 127.11.9.126:36123)
0926 07:40:05.454267 (+ 33502us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:05.454273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:05.454295 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:40:05.454358 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:05.454363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:05.454437 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:05.454573 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:05.456640 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.457901 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.463448 (+ 5547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.464238 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.464376 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.467971 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:05.467979 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:05.467983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:05.468024 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:05.471707 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.475208 (+ 3501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.475216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.475222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.475390 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:05.481323 (+ 5933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:05.481328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:05.481330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:05.481618 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:05.481855 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:05.482851 (+ 996us) server_negotiation.cc:299] Negotiation successful
0926 07:40:05.483104 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33366,"thread_start_us":33241,"threads_started":1}
I20260926 07:40:05.483628 18562 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:05.484035 18562 heartbeater.cc:511] Master 127.11.9.124:33919 requested a full tablet report, sending...
I20260926 07:40:05.485085 18236 ts_manager.cc:194] Registered new tserver with Master: 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:05.485729 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:05.421511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:34813)
0926 07:40:05.448562 (+ 27051us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:05.448614 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:05.450850 (+ 2236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:05.454710 (+ 3860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:05.454718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:05.455289 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:05.455541 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.455554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.460249 (+ 4695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.460254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.462495 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:05.462516 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:05.464429 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:05.464435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:05.464620 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:05.465114 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:05.465139 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:05.467772 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:05.472311 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.472316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.472320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.472816 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.476770 (+ 3954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:05.476775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:05.476779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:05.476979 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:05.481829 (+ 4850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:05.481834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:05.481987 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 07:40:05.482154 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 07:40:05.482460 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26748,"thread_start_us":25947,"threads_started":1}
I20260926 07:40:05.486560 18563 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:05.486699 18563 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:05.487082 18563 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:05.488008 17442 ts_manager.cc:194] Re-registered known tserver with Master: 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:05.488900 17442 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:34813
I20260926 07:40:05.562294 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17771
I20260926 07:40:05.578989 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:36999
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:36123
--webserver_port=44811
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:40:05.854473 18414 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
W20260926 07:40:05.915658 18575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:05.915930 18575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:05.916093 18575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:05.924016 18575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:05.924144 18575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:05.924304 18575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:05.934247 18575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:36999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=44811
--tserver_master_addrs=127.11.9.126:36123,127.11.9.125:32823,127.11.9.124:33919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:05.935600 18575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:05.936966 18575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:05.951175 18580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:05.951109 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:05.951145 18581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:05.951525 18575 server_base.cc:1034] running on GCE node
I20260926 07:40:05.952389 18575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:05.953362 18575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:05.954563 18575 hybrid_clock.cc:648] HybridClock initialized: now 1790408405954495 us; error 55 us; skew 500 ppm
I20260926 07:40:05.954872 18575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:05 dist-test-slave-x492 krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:05.957957 18575 init.cc:377] Logged in from keytab as oryx/127.11.9.67@KRBTEST.COM (short username oryx)
I20260926 07:40:05.959303 18575 webserver.cc:460] Webserver started at http://127.11.9.67:44811/ using document root <none> and password file <none>
I20260926 07:40:05.959983 18575 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:05.960129 18575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:05.965541 18575 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:05.968686 18590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:05.970592 18575 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 07:40:05.970775 18575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "cdd386c92a9d438087d817da3da7d399"
format_stamp: "Formatted at 2026-09-26 07:40:02 on dist-test-slave-x492"
server_key: "e647973adb8f04beee3e4c464ca67a42"
server_key_iv: "e63622261055dd164d0933ff5e5b08ca"
server_key_version: "encryptionkey@0"
I20260926 07:40:05.971815 18575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:05.989681 18575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:05.994060 18575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:05.994356 18575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:05.995270 18575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:05.996636 18575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:05.996749 18575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:05.996889 18575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:05.996990 18575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:06.031529 18575 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:36999
I20260926 07:40:06.031616 18702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:36999 every 8 connection(s)
I20260926 07:40:06.032967 18575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:06.038403 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18575
Sep 26 07:40:06 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.124@KRBTEST.COM
Sep 26 07:40:06 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.125@KRBTEST.COM
I20260926 07:40:06.099478 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.072058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.11.9.126:36123)
0926 07:40:06.073888 (+ 1830us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.073896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.073914 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:40:06.073961 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.073965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.074030 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.074131 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.082589 (+ 8458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.084355 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.086689 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.087513 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.087622 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.091840 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.091849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.091856 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.091903 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.096099 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.096917 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.096922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.096926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.097117 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.097574 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.097579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.097583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.098008 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.098241 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.098816 (+ 575us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.099068 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1664,"thread_start_us":1571,"threads_started":1}
I20260926 07:40:06.099757 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.035865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:34957 (local address 127.11.9.125:32823)
0926 07:40:06.066749 (+ 30884us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.066753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.066770 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:40:06.068723 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.068727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.068791 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.068918 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.073137 (+ 4219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.074181 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.081244 (+ 7063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.082075 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.082193 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.089264 (+ 7071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.089274 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.089282 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.089341 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.093886 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.095851 (+ 1965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.095860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.095865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.096019 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.097803 (+ 1784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.097808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.097812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.098140 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.098256 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.099200 (+ 944us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.099410 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30778,"thread_start_us":30702,"threads_started":1}
I20260926 07:40:06.101152 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.036917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.11.9.67:34957)
0926 07:40:06.063253 (+ 26336us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.063326 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.068277 (+ 4951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.069223 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.069237 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.070327 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:06.071411 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.071446 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.074316 (+ 2870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.074320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.081030 (+ 6710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.081049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.082527 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.082535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.082762 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.085878 (+ 3116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:06.085909 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:06.088948 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:06.094113 (+ 5165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.094125 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.094149 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.094843 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.096173 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.096179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.096184 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.096462 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.098380 (+ 1918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:06.098389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:06.098767 (+ 378us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.099120 (+ 353us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.099654 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25953,"mutex_wait_us":4694,"spinlock_wait_cycles":20864,"thread_start_us":24538,"threads_started":1}
I20260926 07:40:06.102876 18705 heartbeater.cc:347] Connected to a master server at 127.11.9.125:32823
I20260926 07:40:06.103217 18705 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:06.104082 18705 heartbeater.cc:511] Master 127.11.9.125:32823 requested a full tablet report, sending...
I20260926 07:40:06.105319 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.036486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:41297 (local address 127.11.9.124:33919)
0926 07:40:06.065714 (+ 29228us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.065723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.066406 (+ 683us) server_negotiation.cc:407] Connection header received
0926 07:40:06.069221 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.069230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.069332 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.069428 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.071644 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.072664 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.077572 (+ 4908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.078588 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.078722 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.096678 (+ 17956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.096686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.096692 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.096735 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.101719 (+ 4984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.102550 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.102559 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.102564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.102700 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.103358 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.103363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.103366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.103934 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.104064 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.104693 (+ 629us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.104917 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29062,"thread_start_us":28932,"threads_started":1}
I20260926 07:40:06.105751 18047 ts_manager.cc:194] Registered new tserver with Master: cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
I20260926 07:40:06.106525 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.037255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.11.9.67:41297)
0926 07:40:06.066253 (+ 28998us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.066281 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.068295 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.069702 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.069711 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.070738 (+ 1027us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:06.071447 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.071460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.072808 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.072813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.076736 (+ 3923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.076754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.079192 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.079201 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.079600 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.083194 (+ 3594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:06.083233 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:06.095665 (+ 12432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:06.101879 (+ 6214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.101884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.101888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.102403 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.102898 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.102902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.102906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.103165 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.104140 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:06.104145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:06.104375 (+ 230us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.105765 (+ 1390us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.106111 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28847,"mutex_wait_us":108,"thread_start_us":24544,"threads_started":1}
I20260926 07:40:06.107560 18703 heartbeater.cc:347] Connected to a master server at 127.11.9.124:33919
I20260926 07:40:06.107674 18703 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:06.108150 18703 heartbeater.cc:511] Master 127.11.9.124:33919 requested a full tablet report, sending...
I20260926 07:40:06.109287 18236 ts_manager.cc:194] Registered new tserver with Master: cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
Sep 26 07:40:06 dist-test-slave-x492 krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408405, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:06.104476 17442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:06.115571 17442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:06.119172 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.035847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:50861 (local address 127.11.9.126:36123)
0926 07:40:06.061952 (+ 26105us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.061963 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.065201 (+ 3238us) server_negotiation.cc:407] Connection header received
0926 07:40:06.068642 (+ 3441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.068652 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.068763 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.068928 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.071643 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.072851 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.106792 (+ 33941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.107469 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.107592 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.112291 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.112297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.112302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.112352 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.115866 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.116791 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.116797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.116800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.116953 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.117395 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.117400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.117404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.117811 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.117973 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.118531 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.118730 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25923,"thread_start_us":26125,"threads_started":1}
I20260926 07:40:06.119865 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.036474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:50861)
0926 07:40:06.065069 (+ 28595us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.065093 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.068277 (+ 3184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.069223 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.069237 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.070357 (+ 1120us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:06.071447 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.071460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.073006 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.073012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.106664 (+ 33652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.106679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.108349 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.108356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.108588 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.109269 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:06.109294 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:06.112068 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:06.116132 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.116139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.116145 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.116677 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.117079 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:06.117083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:06.117087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:06.117272 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:06.117937 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:06.117941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:06.118165 (+ 224us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.118406 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.118884 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26086,"mutex_wait_us":27487,"thread_start_us":29336,"threads_started":1}
I20260926 07:40:06.121989 18704 heartbeater.cc:347] Connected to a master server at 127.11.9.126:36123
I20260926 07:40:06.122081 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.074133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.11.9.125:32823)
0926 07:40:06.095691 (+ 21558us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.095696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.096256 (+ 560us) server_negotiation.cc:407] Connection header received
0926 07:40:06.096446 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.096453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.096526 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.096628 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.103702 (+ 7074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.104844 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.107534 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.108173 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.108941 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.115061 (+ 6120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.115068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.115072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.115106 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.118689 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.119472 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.119478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.119481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.119627 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.120055 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.120060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.120064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.120395 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.120641 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.121417 (+ 776us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.121630 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21439,"thread_start_us":22281,"threads_started":1}
I20260926 07:40:06.122133 18704 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:06.122502 18704 heartbeater.cc:511] Master 127.11.9.126:36123 requested a full tablet report, sending...
I20260926 07:40:06.123629 17441 ts_manager.cc:194] Re-registered known tserver with Master: cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
I20260926 07:40:06.125378 17441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:50861
I20260926 07:40:06.128293 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.074015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.11.9.124:33919)
0926 07:40:06.093053 (+ 19038us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.093061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.097111 (+ 4050us) server_negotiation.cc:407] Connection header received
0926 07:40:06.097275 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.097281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.097377 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.097470 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.107590 (+ 10120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.108706 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.111187 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.112111 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.112205 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.119856 (+ 7651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.119862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.119867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.119904 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.124706 (+ 4802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.125413 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.125419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.125422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.125550 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.126308 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.126324 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.126330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.126797 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.127149 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.127860 (+ 711us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.128013 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18898,"thread_start_us":19060,"threads_started":1}
I20260926 07:40:06.168264 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.148545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.11.9.67:36999)
0926 07:40:06.158522 (+ 9977us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.158532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.158569 (+ 37us) server_negotiation.cc:407] Connection header received
0926 07:40:06.158880 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.158889 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.159185 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.159451 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.160625 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.163289 (+ 2664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.165709 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.167119 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.167245 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.167458 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.167750 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.167911 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9622,"thread_start_us":9475,"threads_started":1}
I20260926 07:40:06.169117 18735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.148501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36999 (local address 127.0.0.1:43558)
0926 07:40:06.158459 (+ 9958us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.158485 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.158630 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.159808 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.159814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.159846 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.160348 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.160363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.163434 (+ 3071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.163442 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.165508 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.165519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.167350 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.167356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.167565 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.167600 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.167696 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.167905 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9267,"mutex_wait_us":29,"thread_start_us":8942,"threads_started":1}
I20260926 07:40:06.170722 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.158236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:43065 (local address 127.0.0.1:59736)
0926 07:40:06.162752 (+ 4516us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.162766 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.162920 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.163827 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.163832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.163855 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.164236 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.164247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.166109 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.166114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.168375 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.168390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.169950 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.169956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.170121 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.170149 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.170240 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.170372 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":32,"thread_start_us":574,"threads_started":1}
I20260926 07:40:06.171625 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.159089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43457 (local address 127.0.0.1:44344)
0926 07:40:06.162952 (+ 3863us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.162969 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.163685 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.165457 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.165462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.165487 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.165895 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.165905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.167204 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.167211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.169378 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.169391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.171128 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.171132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.171243 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.171263 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.171327 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.171401 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3754,"mutex_wait_us":1,"thread_start_us":3999,"threads_started":1}
I20260926 07:40:06.171823 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.159377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.11.9.65:43457)
0926 07:40:06.162492 (+ 3115us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.162502 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.163067 (+ 565us) server_negotiation.cc:407] Connection header received
0926 07:40:06.163888 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.163895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.164122 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.164292 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.166107 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.167044 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.169550 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.170953 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.171079 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.171261 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.171383 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.171531 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2952,"thread_start_us":3110,"threads_started":1}
I20260926 07:40:06.173440 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.158479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.11.9.66:43065)
0926 07:40:06.161979 (+ 3500us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.161988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.162876 (+ 888us) server_negotiation.cc:407] Connection header received
0926 07:40:06.163154 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.163162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.163408 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.163619 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.164417 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.165979 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.168573 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.169752 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.169885 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.170082 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.170291 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.170448 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3348,"thread_start_us":3246,"threads_started":1}
I20260926 07:40:06.173287 18638 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6c8166a5e4096aa992e7858dce5be (DEFAULT_TABLE table=test-table [id=e9bb98fa580844fbbb1f212762d36d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:06.174690 18638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6c8166a5e4096aa992e7858dce5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:06.174902 18348 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6c8166a5e4096aa992e7858dce5be (DEFAULT_TABLE table=test-table [id=e9bb98fa580844fbbb1f212762d36d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:06.176203 18348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6c8166a5e4096aa992e7858dce5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:06.176620 18497 tablet_service.cc:1467] Processing CreateTablet for tablet 67a6c8166a5e4096aa992e7858dce5be (DEFAULT_TABLE table=test-table [id=e9bb98fa580844fbbb1f212762d36d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:06.177891 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a6c8166a5e4096aa992e7858dce5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:06.186342 18740 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Bootstrap starting.
I20260926 07:40:06.189365 18740 tablet_bootstrap.cc:654] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:06.190984 18740 log.cc:824] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:06.193325 18743 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Bootstrap starting.
I20260926 07:40:06.193878 18740 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: No bootstrap required, opened a new log
I20260926 07:40:06.194137 18740 ts_tablet_manager.cc:1397] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260926 07:40:06.197613 18743 tablet_bootstrap.cc:654] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:06.197952 18741 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Bootstrap starting.
I20260926 07:40:06.199029 18743 log.cc:824] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:06.201612 18741 tablet_bootstrap.cc:654] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:06.202416 18743 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: No bootstrap required, opened a new log
I20260926 07:40:06.202669 18743 ts_tablet_manager.cc:1397] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260926 07:40:06.202802 18741 log.cc:824] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:06.202872 18740 raft_consensus.cc:348] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.203372 18740 raft_consensus.cc:374] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:06.203503 18740 raft_consensus.cc:729] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd386c92a9d438087d817da3da7d399, State: Initialized, Role: FOLLOWER
I20260926 07:40:06.204077 18740 consensus_queue.cc:260] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.205802 18740 ts_tablet_manager.cc:1428] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260926 07:40:06.206073 18704 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
I20260926 07:40:06.206552 18741 tablet_bootstrap.cc:492] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: No bootstrap required, opened a new log
I20260926 07:40:06.206833 18741 ts_tablet_manager.cc:1397] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260926 07:40:06.212585 18743 raft_consensus.cc:348] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.212949 18743 raft_consensus.cc:374] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:06.213069 18743 raft_consensus.cc:729] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22756c7c2632488bb848bd512965b705, State: Initialized, Role: FOLLOWER
I20260926 07:40:06.213738 18743 consensus_queue.cc:260] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.215404 18743 ts_tablet_manager.cc:1428] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260926 07:40:06.215704 18563 heartbeater.cc:503] Master 127.11.9.126:36123 was elected leader, sending a full tablet report...
I20260926 07:40:06.218779 18741 raft_consensus.cc:348] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.219182 18741 raft_consensus.cc:374] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:06.219295 18741 raft_consensus.cc:729] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d656517c6e145fb8bfeacddd154fd03, State: Initialized, Role: FOLLOWER
I20260926 07:40:06.219894 18741 consensus_queue.cc:260] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.221495 18741 ts_tablet_manager.cc:1428] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Time spent starting tablet: real 0.015s user 0.010s sys 0.002s
W20260926 07:40:06.301731 18706 tablet.cc:2369] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:40:06.313057 18416 tablet.cc:2369] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:40:06.319044 18748 raft_consensus.cc:482] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:06.319334 18748 raft_consensus.cc:504] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.320969 18748 leader_election.cc:288] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999), 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:06.320998 18747 raft_consensus.cc:482] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:06.321316 18747 raft_consensus.cc:504] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.329660 18747 leader_election.cc:288] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999), 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457)
I20260926 07:40:06.330866 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.321490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:47865 (local address 127.11.9.67:36999)
0926 07:40:06.321749 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.321758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.323031 (+ 1273us) server_negotiation.cc:407] Connection header received
0926 07:40:06.323406 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.323411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.323481 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.323558 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.324608 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.325690 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.328367 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.329658 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.329758 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.329872 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.330454 (+ 582us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.330597 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 07:40:06.331287 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.322055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:37029 (local address 127.11.9.66:43065)
0926 07:40:06.322297 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.322306 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.323241 (+ 935us) server_negotiation.cc:407] Connection header received
0926 07:40:06.323405 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.323410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.323481 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.323542 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.324392 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.325702 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.328831 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.330167 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.330270 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.330370 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.330931 (+ 561us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.331053 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:40:06.332108 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.321202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36999 (local address 127.11.9.65:47865)
0926 07:40:06.322906 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.322931 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.323174 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.323705 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.323709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.323744 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.324446 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.324460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.325869 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.325874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.328183 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.328202 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.330092 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.330098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.330270 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.330305 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.331556 (+ 1251us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.331694 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1536,"mutex_wait_us":376,"spinlock_wait_cycles":1920,"thread_start_us":1314,"threads_started":1}
I20260926 07:40:06.332463 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.321202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:43065 (local address 127.11.9.65:37029)
0926 07:40:06.323138 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.323153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.323260 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.323710 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.323713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.323750 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.324187 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.324205 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.325869 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.325874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.328665 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.328678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.330593 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.330599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.330755 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.330782 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.331779 (+ 997us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.331904 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1811,"mutex_wait_us":95,"spinlock_wait_cycles":8192,"thread_start_us":1573,"threads_started":1}
I20260926 07:40:06.333130 18658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd386c92a9d438087d817da3da7d399" is_pre_election: true
I20260926 07:40:06.333751 18658 raft_consensus.cc:2454] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 0.
I20260926 07:40:06.333732 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22756c7c2632488bb848bd512965b705" is_pre_election: true
I20260926 07:40:06.334193 18517 raft_consensus.cc:2454] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 0.
I20260926 07:40:06.334512 18301 leader_election.cc:302] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d656517c6e145fb8bfeacddd154fd03, cdd386c92a9d438087d817da3da7d399; no voters:
I20260926 07:40:06.334925 18748 raft_consensus.cc:2781] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:06.335098 18748 raft_consensus.cc:482] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:06.335209 18748 raft_consensus.cc:3037] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:06.338104 18748 raft_consensus.cc:504] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.338963 18748 leader_election.cc:288] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 election: Requested vote from peers cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999), 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065)
I20260926 07:40:06.339190 18658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd386c92a9d438087d817da3da7d399"
I20260926 07:40:06.339344 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22756c7c2632488bb848bd512965b705"
I20260926 07:40:06.339399 18658 raft_consensus.cc:3037] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:06.339538 18517 raft_consensus.cc:3037] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:06.339802 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.323905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40579 (local address 127.11.9.67:36999)
0926 07:40:06.331081 (+ 7176us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.331086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.331101 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:40:06.331150 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.331155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.331258 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.331352 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.333564 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.334759 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.337371 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.338560 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.338707 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.338804 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.339354 (+ 550us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.339503 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7086,"thread_start_us":9326,"threads_started":1}
I20260926 07:40:06.340745 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.323559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36999 (local address 127.11.9.66:40579)
0926 07:40:06.326339 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.326360 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.326485 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.332617 (+ 6132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.332622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.332649 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.333355 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.333367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.334954 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.334963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.337216 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.337231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.338953 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.338961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.339159 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.339206 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.340235 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.340383 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2655,"thread_start_us":2442,"threads_started":1}
I20260926 07:40:06.341864 18657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "22756c7c2632488bb848bd512965b705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd386c92a9d438087d817da3da7d399" is_pre_election: true
I20260926 07:40:06.342646 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.323889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:33945 (local address 127.11.9.65:43457)
0926 07:40:06.324083 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.324090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.333802 (+ 9712us) server_negotiation.cc:407] Connection header received
0926 07:40:06.334205 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.334212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.334297 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.334374 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:06.335561 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.336728 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.339511 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.340614 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.340721 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.340826 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.342334 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.342451 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:40:06.342679 18517 raft_consensus.cc:2454] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 1.
I20260926 07:40:06.342679 18658 raft_consensus.cc:2454] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 1.
I20260926 07:40:06.343072 18302 leader_election.cc:302] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22756c7c2632488bb848bd512965b705, 9d656517c6e145fb8bfeacddd154fd03; no voters:
I20260926 07:40:06.343442 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.323565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43457 (local address 127.11.9.66:33945)
0926 07:40:06.333707 (+ 10142us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:06.333730 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:06.333905 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:06.334834 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:06.334839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:06.334864 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:06.335361 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.335375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.336896 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.336902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.339358 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:06.339371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.341035 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.341041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:06.341252 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.342179 (+ 927us) client_negotiation.cc:769] Sending connection context
0926 07:40:06.342924 (+ 745us) client_negotiation.cc:240] Negotiation successful
0926 07:40:06.343057 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10025,"mutex_wait_us":86,"thread_start_us":10845,"threads_started":1}
I20260926 07:40:06.343444 18748 raft_consensus.cc:2781] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:06.343925 18748 raft_consensus.cc:686] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 1 LEADER]: Becoming Leader. State: Replica: 9d656517c6e145fb8bfeacddd154fd03, State: Running, Role: LEADER
I20260926 07:40:06.344370 18368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a6c8166a5e4096aa992e7858dce5be" candidate_uuid: "22756c7c2632488bb848bd512965b705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d656517c6e145fb8bfeacddd154fd03" is_pre_election: true
I20260926 07:40:06.344517 18748 consensus_queue.cc:237] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } }
I20260926 07:40:06.345438 18451 leader_election.cc:302] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22756c7c2632488bb848bd512965b705; no voters: 9d656517c6e145fb8bfeacddd154fd03, cdd386c92a9d438087d817da3da7d399
I20260926 07:40:06.345835 18747 raft_consensus.cc:2726] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:06.349924 17441 catalog_manager.cc:5495] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65). New cstate: current_term: 1 leader_uuid: "9d656517c6e145fb8bfeacddd154fd03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } health_report { overall_health: HEALTHY } } }
I20260926 07:40:06.399129 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.394536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.11.9.65:43457)
0926 07:40:06.394696 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.394700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.394716 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:40:06.394798 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.394802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.394889 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.395011 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:06.395300 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.396015 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.397254 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.397693 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.397783 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.398174 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:06.398667 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:06.398774 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.398862 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.398920 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:40:06.406659 18658 raft_consensus.cc:1261] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Refusing update from remote peer 9d656517c6e145fb8bfeacddd154fd03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:06.406710 18517 raft_consensus.cc:1261] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Refusing update from remote peer 9d656517c6e145fb8bfeacddd154fd03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:06.407577 18754 consensus_queue.cc:1035] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:06.407943 18748 consensus_queue.cc:1035] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:06.420159 18757 mvcc.cc:205] Tried to move back new op lower bound from 7333512832632930304 to 7333512832391843840. Current Snapshot: MvccSnapshot[applied={T|T < 7333512832632930304}]
W20260926 07:40:06.422151 18565 tablet.cc:2369] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:40:06.430253 18758 mvcc.cc:205] Tried to move back new op lower bound from 7333512832632930304 to 7333512832391843840. Current Snapshot: MvccSnapshot[applied={T|T < 7333512832632930304}]
I20260926 07:40:06.464270 17441 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:06.464576 17441 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:06.470829 17441 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43e5257e3e904e598200dcaf9445dba3: Sending DeleteTablet for 3 replicas of tablet 67a6c8166a5e4096aa992e7858dce5be
I20260926 07:40:06.471896 18638 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6c8166a5e4096aa992e7858dce5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:06 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:43558
I20260926 07:40:06.472744 18767 tablet_replica.cc:330] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: stopping tablet replica
I20260926 07:40:06.472923 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6c8166a5e4096aa992e7858dce5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:06 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:59736
I20260926 07:40:06.473222 18767 raft_consensus.cc:2229] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:06.473292 18348 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a6c8166a5e4096aa992e7858dce5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:06 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:44344
I20260926 07:40:06.473730 18767 raft_consensus.cc:2258] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:06.476389 18767 ts_tablet_manager.cc:1905] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:06.485522 18767 ts_tablet_manager.cc:1918] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:06.485716 18767 log.cc:1196] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/67a6c8166a5e4096aa992e7858dce5be
I20260926 07:40:06.486281 18767 ts_tablet_manager.cc:1939] T 67a6c8166a5e4096aa992e7858dce5be P cdd386c92a9d438087d817da3da7d399: Deleting consensus metadata
I20260926 07:40:06.486328 18769 tablet_replica.cc:330] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: stopping tablet replica
I20260926 07:40:06.486991 18769 raft_consensus.cc:2229] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:06.487668 18769 raft_consensus.cc:2258] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:06.488135 17426 catalog_manager.cc:4841] TS cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999): tablet 67a6c8166a5e4096aa992e7858dce5be (table test-table [id=e9bb98fa580844fbbb1f212762d36d44]) successfully deleted
I20260926 07:40:06.489737 18769 ts_tablet_manager.cc:1905] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:06.491767 18768 tablet_replica.cc:330] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: stopping tablet replica
I20260926 07:40:06.492291 18768 raft_consensus.cc:2229] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:06.492894 18768 raft_consensus.cc:2258] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:06.494721 18768 ts_tablet_manager.cc:1905] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:06.496265 18769 ts_tablet_manager.cc:1918] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:06.496439 18769 log.cc:1196] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/67a6c8166a5e4096aa992e7858dce5be
I20260926 07:40:06.497015 18769 ts_tablet_manager.cc:1939] T 67a6c8166a5e4096aa992e7858dce5be P 9d656517c6e145fb8bfeacddd154fd03: Deleting consensus metadata
I20260926 07:40:06.498744 17427 catalog_manager.cc:4841] TS 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457): tablet 67a6c8166a5e4096aa992e7858dce5be (table test-table [id=e9bb98fa580844fbbb1f212762d36d44]) successfully deleted
I20260926 07:40:06.506605 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17410
I20260926 07:40:06.507004 18768 ts_tablet_manager.cc:1918] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:06.507395 18768 log.cc:1196] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/67a6c8166a5e4096aa992e7858dce5be
I20260926 07:40:06.507732 17472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:06.507988 18768 ts_tablet_manager.cc:1939] T 67a6c8166a5e4096aa992e7858dce5be P 22756c7c2632488bb848bd512965b705: Deleting consensus metadata
I20260926 07:40:06.510193 17427 catalog_manager.cc:4841] TS 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065): tablet 67a6c8166a5e4096aa992e7858dce5be (table test-table [id=e9bb98fa580844fbbb1f212762d36d44]) successfully deleted
I20260926 07:40:06.523880 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.482488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.11.9.125:32823)
0926 07:40:06.482699 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.482706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.505902 (+ 23196us) server_negotiation.cc:407] Connection header received
0926 07:40:06.505987 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.505993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.506080 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.506185 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.509116 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.510245 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.512609 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.513290 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.513423 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.516736 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.516742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.516748 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.516788 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.520359 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.521244 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.521249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.521253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.521405 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.521828 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.521833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.521837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.522167 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.522380 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.522848 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.523553 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 07:40:06.540046 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.506447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.11.9.124:33919)
0926 07:40:06.509010 (+ 2563us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.509016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.523557 (+ 14541us) server_negotiation.cc:407] Connection header received
0926 07:40:06.523783 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.523790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.523911 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.523981 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.525909 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.526986 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.528993 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.529685 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.529807 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.533560 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.533568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.533572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.533609 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.537013 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.537901 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.537907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.537911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.538054 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.538512 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.538517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.538521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.538881 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.539123 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.539441 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.539713 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2414}
I20260926 07:40:06.636161 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17410
I20260926 07:40:06.684566 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.662550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.11.9.124:33919)
0926 07:40:06.662779 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.662788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.665248 (+ 2460us) server_negotiation.cc:407] Connection header received
0926 07:40:06.665870 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.665878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.665988 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.666118 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.668804 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.670471 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.673522 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.674243 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.674349 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.678078 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.678085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.678090 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.678131 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.681593 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.682365 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.682372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.682378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.682552 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.683008 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.683016 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.683022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.683372 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.683620 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.683969 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.684191 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:40:06.686900 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:06.662296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.11.9.125:32823)
0926 07:40:06.664079 (+ 1783us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:06.664088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:06.672879 (+ 8791us) server_negotiation.cc:407] Connection header received
0926 07:40:06.673099 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:06.673103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:06.673202 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:06.673301 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:06.675675 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.676769 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.678536 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:06.678932 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:06.679066 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:06.681465 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:06.681470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:06.681473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:06.681502 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:06.684428 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.685151 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.685156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.685159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.685340 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:06.685720 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:06.685724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:06.685728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:06.686001 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:06.686154 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:06.686508 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 07:40:06.686655 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1607}
I20260926 07:40:07.439834 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.437896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:46661)
0926 07:40:07.439384 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.439554 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1337,"thread_start_us":1228,"threads_started":1}
W20260926 07:40:07.440305 18704 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:36123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.453576 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.452824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:59797)
0926 07:40:07.453302 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.453404 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
W20260926 07:40:07.453981 18414 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:36123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.456720 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.455944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:34719)
0926 07:40:07.456400 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.456524 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
W20260926 07:40:07.457244 18563 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:36123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.971354 18782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43e5257e3e904e598200dcaf9445dba3)
I20260926 07:40:07.971678 18782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } }
I20260926 07:40:07.971997 18783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43e5257e3e904e598200dcaf9445dba3)
I20260926 07:40:07.972273 18783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } }
I20260926 07:40:07.973361 18782 leader_election.cc:288] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123), 7585b7acbba641868d271945a613d6b5 (127.11.9.125:32823)
I20260926 07:40:07.974579 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.973476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49660)
0926 07:40:07.974116 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.974292 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":557,"threads_started":1}
W20260926 07:40:07.974994 18220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.979853 18783 leader_election.cc:288] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123), 15db82b6b33f4accaa139c11b2f3ae83 (127.11.9.124:33919)
I20260926 07:40:07.989904 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.973831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49662)
0926 07:40:07.988615 (+ 14784us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.989542 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14666,"thread_start_us":15278,"threads_started":1}
W20260926 07:40:07.990226 18031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.993573 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.991284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49676)
0926 07:40:07.992863 (+ 1579us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.993063 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1445}
W20260926 07:40:07.993759 18220 leader_election.cc:334] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123): Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.994010 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.992955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49684)
0926 07:40:07.993610 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.993792 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509}
W20260926 07:40:07.994165 18031 leader_election.cc:334] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123): Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:07.995040 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.973585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:32823 (local address 127.0.0.1:37542)
0926 07:40:07.980587 (+ 7002us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.980614 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:07.980791 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:07.985279 (+ 4488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:07.985287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:07.985320 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:07.985619 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:07.985634 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.987740 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.987750 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:07.990703 (+ 2953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:07.990727 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.994488 (+ 3761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.994493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:07.994637 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:07.994662 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 07:40:07.994753 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 07:40:07.994859 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6854,"thread_start_us":19002,"threads_started":1}
I20260926 07:40:07.995333 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.973830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.11.9.125:32823)
0926 07:40:07.984937 (+ 11107us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:07.984944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:07.984967 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:40:07.985040 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:07.985048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:07.985126 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:07.985267 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:07.986378 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.987563 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.990881 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.994321 (+ 3440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.994442 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:07.994538 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:07.994812 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 07:40:07.994972 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10970,"thread_start_us":12817,"threads_started":1}
I20260926 07:40:07.996037 18057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15db82b6b33f4accaa139c11b2f3ae83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7585b7acbba641868d271945a613d6b5" is_pre_election: true
I20260926 07:40:07.996352 18057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15db82b6b33f4accaa139c11b2f3ae83 in term 1.
I20260926 07:40:07.996850 18223 leader_election.cc:302] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15db82b6b33f4accaa139c11b2f3ae83, 7585b7acbba641868d271945a613d6b5; no voters: 43e5257e3e904e598200dcaf9445dba3
I20260926 07:40:07.997355 18782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 07:40:07.997512 18782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43e5257e3e904e598200dcaf9445dba3)
I20260926 07:40:07.997632 18782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:40:07.999269 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.987236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.11.9.124:33919)
0926 07:40:07.992536 (+ 5300us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:07.992549 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:07.992582 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:40:07.992654 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:07.992661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:07.992758 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:07.992869 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:07.993601 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.994823 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.997263 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.998371 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.998483 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:07.998567 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:07.998902 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 07:40:07.999002 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5038,"thread_start_us":4893,"threads_started":1}
I20260926 07:40:07.999306 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:07.991813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:33919 (local address 127.0.0.1:59304)
0926 07:40:07.992130 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:07.992149 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:07.992269 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:07.992902 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:07.992907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:07.992953 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:07.993401 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:07.993414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.994977 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.994982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:07.996587 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:07.996600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:07.998554 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:07.998560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:07.998732 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:07.998777 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 07:40:07.998878 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 07:40:07.999019 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":1507,"threads_started":1}
I20260926 07:40:07.999944 18246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7585b7acbba641868d271945a613d6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "15db82b6b33f4accaa139c11b2f3ae83" is_pre_election: true
I20260926 07:40:08.000622 18782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } }
I20260926 07:40:08.001046 18246 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7585b7acbba641868d271945a613d6b5 in current term 2: Already voted for candidate 15db82b6b33f4accaa139c11b2f3ae83 in this term.
I20260926 07:40:08.001353 18782 leader_election.cc:288] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 election: Requested vote from peers 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123), 7585b7acbba641868d271945a613d6b5 (127.11.9.125:32823)
I20260926 07:40:08.001837 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.001437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49700)
0926 07:40:08.001604 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.001695 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 07:40:08.001802 18032 leader_election.cc:302] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7585b7acbba641868d271945a613d6b5; no voters: 15db82b6b33f4accaa139c11b2f3ae83, 43e5257e3e904e598200dcaf9445dba3
I20260926 07:40:08.002143 18783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:40:08.002853 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.002472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49704)
0926 07:40:08.002663 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.002748 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 07:40:08.002799 18057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15db82b6b33f4accaa139c11b2f3ae83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7585b7acbba641868d271945a613d6b5"
W20260926 07:40:08.003057 18220 leader_election.cc:334] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123): Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
I20260926 07:40:08.004607 18783 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:40:08.006801 18057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15db82b6b33f4accaa139c11b2f3ae83 in term 2.
I20260926 07:40:08.007534 18223 leader_election.cc:302] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15db82b6b33f4accaa139c11b2f3ae83, 7585b7acbba641868d271945a613d6b5; no voters: 43e5257e3e904e598200dcaf9445dba3
I20260926 07:40:08.007843 18782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:40:08.008284 18782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [term 2 LEADER]: Becoming Leader. State: Replica: 15db82b6b33f4accaa139c11b2f3ae83, State: Running, Role: LEADER
I20260926 07:40:08.008716 18782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } }
I20260926 07:40:08.011132 18792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15db82b6b33f4accaa139c11b2f3ae83. Latest consensus state: current_term: 2 leader_uuid: "15db82b6b33f4accaa139c11b2f3ae83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:08.011555 18792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:08.012193 18794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:08.015609 18794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:08.016417 18794 catalog_manager.cc:1259] Loaded cluster ID: 65811b5f4021461f985a334f6a402d4e
I20260926 07:40:08.016525 18794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:08.017796 18794 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:08.018723 18794 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Loaded TSK: 0
I20260926 07:40:08.019606 18794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:40:08.067576 18236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:08.074340 18236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:08.078464 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.078020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49732)
0926 07:40:08.078174 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.078284 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 07:40:08.078576 18057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [term 2 FOLLOWER]: Refusing update from remote peer 15db82b6b33f4accaa139c11b2f3ae83: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 07:40:08.079447 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.079067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49746)
0926 07:40:08.079218 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.079307 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 07:40:08.079397 18792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 07:40:08.079809 18220 consensus_peers.cc:487] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 -> Peer 43e5257e3e904e598200dcaf9445dba3 (127.11.9.126:36123): Couldn't send request to peer 43e5257e3e904e598200dcaf9445dba3. Status: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:40:08.081839 18783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15db82b6b33f4accaa139c11b2f3ae83. Latest consensus state: current_term: 2 leader_uuid: "15db82b6b33f4accaa139c11b2f3ae83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:08.082135 18783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:08.083578 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "15db82b6b33f4accaa139c11b2f3ae83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:08.083904 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:08.084827 18783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "15db82b6b33f4accaa139c11b2f3ae83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "43e5257e3e904e598200dcaf9445dba3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 36123 } } peers { permanent_uuid: "7585b7acbba641868d271945a613d6b5" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 32823 } attrs { promote: false } } peers { permanent_uuid: "15db82b6b33f4accaa139c11b2f3ae83" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 33919 } attrs { promote: false } } }
I20260926 07:40:08.085127 18783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7585b7acbba641868d271945a613d6b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:40:08.103041 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.092003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43457 (local address 127.0.0.1:44352)
0926 07:40:08.092186 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.092215 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:08.092318 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:08.093228 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:08.093232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:08.093259 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:08.093639 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.093653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.095541 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.095550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.098351 (+ 2801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.098372 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.102263 (+ 3891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.102274 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.102503 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.102544 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 07:40:08.102608 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 07:40:08.102754 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":26}
I20260926 07:40:08.103776 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.092136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.11.9.65:43457)
0926 07:40:08.092769 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.092778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.092801 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:40:08.092925 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.092933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.093027 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.093802 (+ 775us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:08.093999 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.095345 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.100233 (+ 4888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.101943 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.103385 (+ 1442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.103456 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.103495 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.103588 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":481,"threads_started":1}
I20260926 07:40:08.104466 18348 tablet_service.cc:1467] Processing CreateTablet for tablet e6fc475609e54fdbb4de253be5944ecb (DEFAULT_TABLE table=test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:08.104612 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.091272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:43065 (local address 127.0.0.1:59740)
0926 07:40:08.091525 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.091546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:08.091651 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:08.092208 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:08.092213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:08.092241 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:08.092491 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.092502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.094449 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.094454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.099977 (+ 5523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.099994 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.103603 (+ 3609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.103612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.103880 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.103930 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 07:40:08.104005 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 07:40:08.104193 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2821}
I20260926 07:40:08.104874 18348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6fc475609e54fdbb4de253be5944ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:08.105660 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.091415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.11.9.66:43065)
0926 07:40:08.091866 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.091871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.091886 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:40:08.091977 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.091981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.092054 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.092650 (+ 596us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:08.092850 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.094279 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.100197 (+ 5918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.103385 (+ 3188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.105061 (+ 1676us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.105218 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.105301 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.105447 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":1095,"threads_started":1}
I20260926 07:40:08.106408 18497 tablet_service.cc:1467] Processing CreateTablet for tablet e6fc475609e54fdbb4de253be5944ecb (DEFAULT_TABLE table=test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:08.106844 18497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6fc475609e54fdbb4de253be5944ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:08.112471 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.092377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:36999 (local address 127.0.0.1:43566)
0926 07:40:08.102512 (+ 10135us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.102529 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:08.102631 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:08.106157 (+ 3526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:08.106161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:08.106186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:08.106478 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.106489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.107798 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.107803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.109858 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:08.109873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.111717 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.111723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:08.111900 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.111946 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 07:40:08.112039 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 07:40:08.112161 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10009,"thread_start_us":10310,"threads_started":1}
I20260926 07:40:08.112655 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.092503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.11.9.67:36999)
0926 07:40:08.105550 (+ 13047us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.105556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.105575 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:40:08.105639 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.105644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.105722 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.106127 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:08.106638 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.107671 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.110014 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.111481 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.111656 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.111808 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.112089 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.112275 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12922,"thread_start_us":12833,"threads_started":1}
I20260926 07:40:08.113610 18638 tablet_service.cc:1467] Processing CreateTablet for tablet e6fc475609e54fdbb4de253be5944ecb (DEFAULT_TABLE table=test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:08.113721 18806 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Bootstrap starting.
I20260926 07:40:08.113996 18638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6fc475609e54fdbb4de253be5944ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:08.116040 18806 tablet_bootstrap.cc:654] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:08.117008 18805 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Bootstrap starting.
I20260926 07:40:08.118409 18806 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: No bootstrap required, opened a new log
I20260926 07:40:08.118562 18806 ts_tablet_manager.cc:1397] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260926 07:40:08.118829 18805 tablet_bootstrap.cc:654] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:08.119402 18806 raft_consensus.cc:348] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.119637 18806 raft_consensus.cc:374] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:08.119729 18806 raft_consensus.cc:729] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22756c7c2632488bb848bd512965b705, State: Initialized, Role: FOLLOWER
I20260926 07:40:08.119997 18806 consensus_queue.cc:260] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.120817 18806 ts_tablet_manager.cc:1428] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 07:40:08.122172 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.121322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:41073)
0926 07:40:08.121854 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.121965 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"threads_started":1}
I20260926 07:40:08.124073 18562 heartbeater.cc:503] Master 127.11.9.124:33919 was elected leader, sending a full tablet report...
I20260926 07:40:08.125480 18808 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Bootstrap starting.
I20260926 07:40:08.126333 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.124882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:56407)
0926 07:40:08.126093 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.126181 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1114}
I20260926 07:40:08.127199 18808 tablet_bootstrap.cc:654] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:08.129047 18805 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: No bootstrap required, opened a new log
I20260926 07:40:08.129245 18805 ts_tablet_manager.cc:1397] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.004s
I20260926 07:40:08.129489 18808 tablet_bootstrap.cc:492] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: No bootstrap required, opened a new log
I20260926 07:40:08.129621 18808 ts_tablet_manager.cc:1397] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 07:40:08.130016 18805 raft_consensus.cc:348] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.130209 18805 raft_consensus.cc:374] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:08.130300 18805 raft_consensus.cc:729] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d656517c6e145fb8bfeacddd154fd03, State: Initialized, Role: FOLLOWER
I20260926 07:40:08.130241 18808 raft_consensus.cc:348] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.130460 18808 raft_consensus.cc:374] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:08.130569 18808 raft_consensus.cc:729] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd386c92a9d438087d817da3da7d399, State: Initialized, Role: FOLLOWER
I20260926 07:40:08.130592 18805 consensus_queue.cc:260] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.130860 18808 consensus_queue.cc:260] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.131475 18805 ts_tablet_manager.cc:1428] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:08.131642 18808 ts_tablet_manager.cc:1428] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:08.134056 18703 heartbeater.cc:503] Master 127.11.9.124:33919 was elected leader, sending a full tablet report...
I20260926 07:40:08.134187 18413 heartbeater.cc:503] Master 127.11.9.124:33919 was elected leader, sending a full tablet report...
I20260926 07:40:08.136634 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.135787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:53897)
0926 07:40:08.136265 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.136402 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"threads_started":1}
I20260926 07:40:08.138059 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.131973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:53387)
0926 07:40:08.137645 (+ 5672us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.137752 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5548,"thread_start_us":5440,"threads_started":1}
I20260926 07:40:08.139242 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.138897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:58577)
0926 07:40:08.139078 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.139141 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 07:40:08.141397 18813 raft_consensus.cc:482] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:08.141526 18813 raft_consensus.cc:504] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.142059 18813 leader_election.cc:288] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065), cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
I20260926 07:40:08.142485 18658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6fc475609e54fdbb4de253be5944ecb" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd386c92a9d438087d817da3da7d399" is_pre_election: true
I20260926 07:40:08.142498 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6fc475609e54fdbb4de253be5944ecb" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22756c7c2632488bb848bd512965b705" is_pre_election: true
I20260926 07:40:08.142709 18658 raft_consensus.cc:2454] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 0.
I20260926 07:40:08.142747 18517 raft_consensus.cc:2454] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 0.
I20260926 07:40:08.143066 18301 leader_election.cc:302] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d656517c6e145fb8bfeacddd154fd03, cdd386c92a9d438087d817da3da7d399; no voters:
I20260926 07:40:08.143440 18813 raft_consensus.cc:2781] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:08.143532 18813 raft_consensus.cc:482] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:08.143620 18813 raft_consensus.cc:3037] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:08.145766 18813 raft_consensus.cc:504] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.146188 18813 leader_election.cc:288] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 election: Requested vote from peers 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065), cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999)
I20260926 07:40:08.146667 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6fc475609e54fdbb4de253be5944ecb" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22756c7c2632488bb848bd512965b705"
I20260926 07:40:08.146667 18658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6fc475609e54fdbb4de253be5944ecb" candidate_uuid: "9d656517c6e145fb8bfeacddd154fd03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd386c92a9d438087d817da3da7d399"
I20260926 07:40:08.146862 18517 raft_consensus.cc:3037] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:08.146869 18658 raft_consensus.cc:3037] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:08.149765 18517 raft_consensus.cc:2454] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 1.
I20260926 07:40:08.149765 18658 raft_consensus.cc:2454] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d656517c6e145fb8bfeacddd154fd03 in term 1.
I20260926 07:40:08.150166 18301 leader_election.cc:302] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d656517c6e145fb8bfeacddd154fd03, cdd386c92a9d438087d817da3da7d399; no voters:
I20260926 07:40:08.150465 18813 raft_consensus.cc:2781] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:08.150882 18813 raft_consensus.cc:686] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 1 LEADER]: Becoming Leader. State: Replica: 9d656517c6e145fb8bfeacddd154fd03, State: Running, Role: LEADER
I20260926 07:40:08.151176 18813 consensus_queue.cc:237] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } }
I20260926 07:40:08.151863 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.151423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:55393)
0926 07:40:08.151655 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.151741 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260926 07:40:08.153946 18235 catalog_manager.cc:5495] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65). New cstate: current_term: 1 leader_uuid: "9d656517c6e145fb8bfeacddd154fd03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d656517c6e145fb8bfeacddd154fd03" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260926 07:40:08.182417 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.177963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.11.9.65:43457)
0926 07:40:08.178161 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.178167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.178185 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:40:08.178339 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.178344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.178420 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.178514 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:08.178862 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.179545 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.180736 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.181143 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.181349 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.181730 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:08.181840 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:08.181940 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.182089 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.182159 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:40:08.185122 18658 raft_consensus.cc:1261] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Refusing update from remote peer 9d656517c6e145fb8bfeacddd154fd03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:08.185252 18517 raft_consensus.cc:1261] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Refusing update from remote peer 9d656517c6e145fb8bfeacddd154fd03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:08.185837 18817 consensus_queue.cc:1035] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd386c92a9d438087d817da3da7d399" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:08.186159 18813 consensus_queue.cc:1035] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22756c7c2632488bb848bd512965b705" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:08.189056 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.188299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:51661)
0926 07:40:08.188628 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.188754 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260926 07:40:08.190119 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.188313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:36437)
0926 07:40:08.189756 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.189891 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1285}
I20260926 07:40:08.196388 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.195830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.66:47827)
0926 07:40:08.196129 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.196211 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260926 07:40:08.196899 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.196449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.67:38891)
0926 07:40:08.196638 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.196706 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 07:40:08.204639 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.204230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:45979)
0926 07:40:08.204388 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.204483 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 07:40:08.208268 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.205325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.11.9.65:40383)
0926 07:40:08.208008 (+ 2683us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.208104 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2605}
I20260926 07:40:08.209343 18235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:08.209584 18235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:08.214753 18235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15db82b6b33f4accaa139c11b2f3ae83: Sending DeleteTablet for 3 replicas of tablet e6fc475609e54fdbb4de253be5944ecb
I20260926 07:40:08.215656 18497 tablet_service.cc:1514] Processing DeleteTablet for tablet e6fc475609e54fdbb4de253be5944ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:08 UTC) from {username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:59740
I20260926 07:40:08.215711 18348 tablet_service.cc:1514] Processing DeleteTablet for tablet e6fc475609e54fdbb4de253be5944ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:08 UTC) from {username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:44352
I20260926 07:40:08.215802 18638 tablet_service.cc:1514] Processing DeleteTablet for tablet e6fc475609e54fdbb4de253be5944ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:08 UTC) from {username='oryx', principal='oryx/127.11.9.124@KRBTEST.COM'} at 127.0.0.1:43566
I20260926 07:40:08.217497 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18200
I20260926 07:40:08.228190 18830 tablet_replica.cc:330] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: stopping tablet replica
I20260926 07:40:08.228464 18830 raft_consensus.cc:2229] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:08.229039 18830 raft_consensus.cc:2258] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:08.229707 18829 tablet_replica.cc:330] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: stopping tablet replica
I20260926 07:40:08.229902 18829 raft_consensus.cc:2229] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:08.230182 18829 raft_consensus.cc:2258] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:08.230335 18830 ts_tablet_manager.cc:1905] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:08.231105 18831 tablet_replica.cc:330] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: stopping tablet replica
I20260926 07:40:08.231266 18829 ts_tablet_manager.cc:1905] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:08.231305 18831 raft_consensus.cc:2229] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:08.231618 18831 raft_consensus.cc:2258] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:08.232707 18831 ts_tablet_manager.cc:1905] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:08.236953 18830 ts_tablet_manager.cc:1918] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:08.237093 18830 log.cc:1196] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/e6fc475609e54fdbb4de253be5944ecb
I20260926 07:40:08.237558 18830 ts_tablet_manager.cc:1939] T e6fc475609e54fdbb4de253be5944ecb P 9d656517c6e145fb8bfeacddd154fd03: Deleting consensus metadata
I20260926 07:40:08.238162 18829 ts_tablet_manager.cc:1918] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:08.238281 18829 log.cc:1196] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/e6fc475609e54fdbb4de253be5944ecb
I20260926 07:40:08.238690 18829 ts_tablet_manager.cc:1939] T e6fc475609e54fdbb4de253be5944ecb P 22756c7c2632488bb848bd512965b705: Deleting consensus metadata
I20260926 07:40:08.238792 18221 catalog_manager.cc:4841] TS 9d656517c6e145fb8bfeacddd154fd03 (127.11.9.65:43457): tablet e6fc475609e54fdbb4de253be5944ecb (table test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]) successfully deleted
I20260926 07:40:08.239164 18831 ts_tablet_manager.cc:1918] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:08.239288 18831 log.cc:1196] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/e6fc475609e54fdbb4de253be5944ecb
I20260926 07:40:08.239698 18831 ts_tablet_manager.cc:1939] T e6fc475609e54fdbb4de253be5944ecb P cdd386c92a9d438087d817da3da7d399: Deleting consensus metadata
I20260926 07:40:08.239705 18221 catalog_manager.cc:4841] TS 22756c7c2632488bb848bd512965b705 (127.11.9.66:43065): tablet e6fc475609e54fdbb4de253be5944ecb (table test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]) successfully deleted
I20260926 07:40:08.240837 18220 catalog_manager.cc:4841] TS cdd386c92a9d438087d817da3da7d399 (127.11.9.67:36999): tablet e6fc475609e54fdbb4de253be5944ecb (table test-table [id=d08ddb4ed4d54facb4af5e97abc728c2]) successfully deleted
I20260926 07:40:08.246627 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.219619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.11.9.65:43457)
0926 07:40:08.219827 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.219835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.229755 (+ 9920us) server_negotiation.cc:407] Connection header received
0926 07:40:08.230475 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.230483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.230600 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.230710 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:08.233239 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.234371 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.236229 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.236917 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.237016 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.241511 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:08.241525 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:08.241541 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:08.241571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:08.244123 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.244786 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.244793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.244795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.244942 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.245329 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.245336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.245342 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.245689 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:08.245914 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.246125 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.246270 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:40:08.247282 18408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:08.365653 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18200
I20260926 07:40:08.387600 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18434
I20260926 07:40:08.400738 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.388297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.11.9.66:43065)
0926 07:40:08.388514 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.388521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.388619 (+ 98us) server_negotiation.cc:407] Connection header received
0926 07:40:08.388822 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.388827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.388895 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.389008 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:08.390551 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.391232 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.392494 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.392889 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.393025 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.395380 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:08.395393 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:08.395409 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:08.395440 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:08.398111 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.398793 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.398801 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.398803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.398925 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.399297 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.399303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.399307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.399656 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:08.400001 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.400209 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.400427 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:40:08.401576 18557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:08.491192 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.490565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49750)
0926 07:40:08.490984 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.491068 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 07:40:08.492182 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.491872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:36123 (local address 127.0.0.1:49760)
0926 07:40:08.492001 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:08.492071 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:36123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 07:40:08.496366 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18434
I20260926 07:40:08.514405 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18575
I20260926 07:40:08.527104 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:08.515133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.11.9.67:36999)
0926 07:40:08.515304 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:08.515308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:08.515326 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:40:08.515488 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:08.515492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:08.515549 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:08.515628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:08.516870 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.517655 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.518881 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:08.519302 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:08.519405 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:08.522000 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:08.522013 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:08.522030 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:08.522059 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:08.524794 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.525437 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.525444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.525446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.525554 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:08.525847 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:08.525850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:08.525851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:08.526178 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:08.526399 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:08.526611 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 07:40:08.526757 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 07:40:08.527753 18698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:08.621197 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18575
I20260926 07:40:08.638774 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 17904
I20260926 07:40:08.639868 18077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:08.754787 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 17904
I20260926 07:40:08.775003 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18003
I20260926 07:40:08.776026 18266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:08.897974 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18003
2026-09-26T07:40:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9343 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:40:09 dist-test-slave-x492 krb5kdc[18840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:40:09 dist-test-slave-x492 krb5kdc[18840](info): set up 2 sockets
Sep 26 07:40:09 dist-test-slave-x492 krb5kdc[18840](info): commencing operation
krb5kdc: starting...
W20260926 07:40:10.002168 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:10 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab.
Entry for principal oryx/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab.
I20260926 07:40:10.203930 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:40473
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab
--principal=oryx/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:40473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:10.518196 18853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:10.518460 18853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:10.518620 18853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:10.525583 18853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:10.525681 18853 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 07:40:10.525794 18853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:10.525907 18853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:10.525993 18853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:10.526108 18853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:10.535766 18853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:40473
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.126.keytab
--principal=oryx/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:40473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:10.537032 18853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:10.538354 18853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:10.549700 18858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:10.549700 18859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:10.549928 18861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:10.550642 18853 server_base.cc:1034] running on GCE node
I20260926 07:40:10.551330 18853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:10.552606 18853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:10.553862 18853 hybrid_clock.cc:648] HybridClock initialized: now 1790408410553807 us; error 67 us; skew 500 ppm
I20260926 07:40:10.554186 18853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:10 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408410, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:10.557282 18853 init.cc:377] Logged in from keytab as oryx/127.11.9.126@KRBTEST.COM (short username oryx)
I20260926 07:40:10.558681 18853 webserver.cc:460] Webserver started at http://127.11.9.126:41923/ using document root <none> and password file <none>
I20260926 07:40:10.559346 18853 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:10.559473 18853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:10.559787 18853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:10.562672 18853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c"
format_stamp: "Formatted at 2026-09-26 07:40:10 on dist-test-slave-x492"
server_key: "e1066869b673551901b3400ca9e80c0c"
server_key_iv: "fb71d92f093e4fae4f50d4b65c311a05"
server_key_version: "encryptionkey@0"
I20260926 07:40:10.563447 18853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c"
format_stamp: "Formatted at 2026-09-26 07:40:10 on dist-test-slave-x492"
server_key: "e1066869b673551901b3400ca9e80c0c"
server_key_iv: "fb71d92f093e4fae4f50d4b65c311a05"
server_key_version: "encryptionkey@0"
I20260926 07:40:10.568315 18853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 07:40:10.571769 18868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:10.573036 18853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:40:10.573289 18853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c"
format_stamp: "Formatted at 2026-09-26 07:40:10 on dist-test-slave-x492"
server_key: "e1066869b673551901b3400ca9e80c0c"
server_key_iv: "fb71d92f093e4fae4f50d4b65c311a05"
server_key_version: "encryptionkey@0"
I20260926 07:40:10.573508 18853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:10.600296 18853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:10.604344 18853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:10.604633 18853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:10.625708 18853 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:40473
I20260926 07:40:10.625728 18929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:40473 every 8 connection(s)
I20260926 07:40:10.627043 18853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:10.630640 18930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:10.632220 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18853
I20260926 07:40:10.632422 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:10.633246 11301 external_mini_cluster.cc:1428] Setting key cb2c42439c597f332b996a2683c22626
I20260926 07:40:10.644533 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Bootstrap starting.
Sep 26 07:40:10 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:10.648240 18930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:10.649354 18930 log.cc:824] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:10.651971 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: No bootstrap required, opened a new log
I20260926 07:40:10.655591 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:10.635313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.11.9.126:40473)
0926 07:40:10.636105 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:10.636115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:10.636207 (+ 92us) server_negotiation.cc:407] Connection header received
0926 07:40:10.637860 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:10.637881 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:10.638239 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:10.638541 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:10.640188 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:10.641543 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:10.643371 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:10.644005 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:10.644135 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:10.648633 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:10.648640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:10.648654 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:10.648689 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:10.651668 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:10.652396 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:10.652403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:10.652408 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:10.652583 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:10.652969 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:10.652975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:10.652989 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:10.653525 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:10.653833 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:10.654276 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 07:40:10.654686 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":400,"threads_started":1}
I20260926 07:40:10.659585 18930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } }
I20260926 07:40:10.659917 18930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:10.660022 18930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b5eaea124ca4b7bbf5cfc0d74abbe8c, State: Initialized, Role: FOLLOWER
I20260926 07:40:10.660526 18930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } }
I20260926 07:40:10.660758 18930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:10.660895 18930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:10.661064 18930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:10.663131 18930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } }
I20260926 07:40:10.663653 18930 leader_election.cc:302] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b5eaea124ca4b7bbf5cfc0d74abbe8c; no voters:
I20260926 07:40:10.664291 18930 leader_election.cc:288] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:10.664387 18935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:10.665131 18935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [term 1 LEADER]: Becoming Leader. State: Replica: 5b5eaea124ca4b7bbf5cfc0d74abbe8c, State: Running, Role: LEADER
I20260926 07:40:10.665774 18935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } }
I20260926 07:40:10.666260 18930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:10.668854 18937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b5eaea124ca4b7bbf5cfc0d74abbe8c. Latest consensus state: current_term: 1 leader_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } } }
I20260926 07:40:10.669554 18937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:10.669108 18936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b5eaea124ca4b7bbf5cfc0d74abbe8c" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40473 } } }
I20260926 07:40:10.670039 18936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:10.671521 18943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:10.677377 18943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:10.687811 18943 catalog_manager.cc:1347] Generated new cluster ID: 04fb91a1e9e949d0ab23abcc4144c5f7
I20260926 07:40:10.687935 18943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:10.698717 18943 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:10.699993 18943 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:10.710387 18943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Generated new TSK 0
I20260926 07:40:10.711213 18943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
I20260926 07:40:10.774150 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40473
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:11.123169 18956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:11.123423 18956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:11.123576 18956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:11.130770 18956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:11.130908 18956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:11.131072 18956 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:40:11.131203 18956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:11.140970 18956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:40473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:11.142211 18956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:11.143507 18956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:11.156567 18962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:11.156513 18964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:11.156831 18961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:11.157495 18956 server_base.cc:1034] running on GCE node
I20260926 07:40:11.158115 18956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:11.159034 18956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:11.160274 18956 hybrid_clock.cc:648] HybridClock initialized: now 1790408411160201 us; error 58 us; skew 500 ppm
I20260926 07:40:11.160579 18956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:11 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408411, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:11.163642 18956 init.cc:377] Logged in from keytab as oryx/127.11.9.65@KRBTEST.COM (short username oryx)
I20260926 07:40:11.165017 18956 webserver.cc:460] Webserver started at http://127.11.9.65:33967/ using document root <none> and password file <none>
I20260926 07:40:11.165721 18956 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:11.165845 18956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:11.166152 18956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:11.169056 18956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "7d235aea3d65ea222e8c0c9943ab2ea9"
server_key_iv: "24e2f4416a906ce671c85196b4b4aba6"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.169868 18956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "7d235aea3d65ea222e8c0c9943ab2ea9"
server_key_iv: "24e2f4416a906ce671c85196b4b4aba6"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.174854 18956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:40:11.178375 18971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:11.179596 18956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:40:11.179762 18956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "7d235aea3d65ea222e8c0c9943ab2ea9"
server_key_iv: "24e2f4416a906ce671c85196b4b4aba6"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.179972 18956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:11.196141 18956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:11.200258 18956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:11.200525 18956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:11.203191 18956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:11.203308 18956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:11.203465 18956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:11.203544 18956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:40:11 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408411, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:11.228585 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.204404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.65:50081)
0926 07:40:11.205373 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:11.205427 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:11.207302 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:11.208085 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:11.208093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:11.208884 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:11.210094 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.210123 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.211819 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.211824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.214636 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.214657 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.215599 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.215608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.215993 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.216874 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:11.216916 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:11.219934 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:11.223177 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.223191 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.223210 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.223793 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.224308 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.224316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.224320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.224665 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.225435 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:11.225441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:11.225951 (+ 510us) client_negotiation.cc:769] Sending connection context
0926 07:40:11.226319 (+ 368us) client_negotiation.cc:240] Negotiation successful
0926 07:40:11.226851 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":400,"threads_started":1}
I20260926 07:40:11.229995 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.203679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:50081 (local address 127.11.9.126:40473)
0926 07:40:11.204439 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:11.204445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:11.205546 (+ 1101us) server_negotiation.cc:407] Connection header received
0926 07:40:11.207576 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:11.207585 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:11.207696 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:11.207819 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:11.210302 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.211602 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.214805 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.215312 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.215460 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.220169 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:11.220174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:11.220177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:11.220207 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:11.223039 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.224012 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.224018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.224023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.224174 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.224824 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.224827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.224829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.225208 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:11.225363 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:11.229410 (+ 4047us) server_negotiation.cc:299] Negotiation successful
0926 07:40:11.229659 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":523,"threads_started":1}
I20260926 07:40:11.253618 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.238730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:57871 (local address 127.11.9.126:40473)
0926 07:40:11.238949 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:11.238958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:11.239887 (+ 929us) server_negotiation.cc:407] Connection header received
0926 07:40:11.240110 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:11.240117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:11.240201 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:11.240332 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:11.241760 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.242838 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.244212 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.244627 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.244768 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.247170 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:11.247175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:11.247178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:11.247207 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:11.250473 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.251408 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.251416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.251420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.251587 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.252113 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.252118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.252121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.252399 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:11.252529 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:11.253116 (+ 587us) server_negotiation.cc:299] Negotiation successful
0926 07:40:11.253368 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:40:11.254151 19057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.238739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.65:57871)
0926 07:40:11.239749 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:11.239780 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:11.239955 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:11.240355 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:11.240360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:11.241022 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:11.241587 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.241608 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.242984 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.242989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.244061 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.244072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.244811 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.244817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.244980 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.245707 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:11.245722 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:11.246913 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:11.250646 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.250650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.250652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.251269 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.251729 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.251736 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.251741 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.251979 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.252542 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:11.252549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:11.252784 (+ 235us) client_negotiation.cc:769] Sending connection context
0926 07:40:11.252988 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 07:40:11.253481 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":631,"threads_started":1}
I20260926 07:40:11.258359 18956 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:45277
I20260926 07:40:11.258383 19091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:45277 every 8 connection(s)
I20260926 07:40:11.259553 18956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:11.261340 19092 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40473
I20260926 07:40:11.261574 19092 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:11.262223 19092 heartbeater.cc:511] Master 127.11.9.126:40473 requested a full tablet report, sending...
I20260926 07:40:11.264063 18885 ts_manager.cc:194] Registered new tserver with Master: fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277)
I20260926 07:40:11.265589 18885 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:50081
I20260926 07:40:11.267241 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 18956
I20260926 07:40:11.267436 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:11.268143 11301 external_mini_cluster.cc:1428] Setting key 570970c0174fc00804a626b369810483
WARNING: no policy specified for oryx/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
I20260926 07:40:11.312249 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40473
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:11.625367 19096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:11.625656 19096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:11.625857 19096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:11.633013 19096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:11.633137 19096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:11.633308 19096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:40:11.633447 19096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:11.642930 19096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:40473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:11.644155 19096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:11.645489 19096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:11.658840 19101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:11.658684 19102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:11.658650 19104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:11.659426 19096 server_base.cc:1034] running on GCE node
I20260926 07:40:11.659987 19096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:11.660822 19096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:11.662053 19096 hybrid_clock.cc:648] HybridClock initialized: now 1790408411661996 us; error 60 us; skew 500 ppm
I20260926 07:40:11.662336 19096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:11 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408411, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:11.665390 19096 init.cc:377] Logged in from keytab as oryx/127.11.9.66@KRBTEST.COM (short username oryx)
I20260926 07:40:11.666652 19096 webserver.cc:460] Webserver started at http://127.11.9.66:41205/ using document root <none> and password file <none>
I20260926 07:40:11.667371 19096 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:11.667476 19096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:11.667789 19096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:11.670691 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "50d613922f644c4baf64a1efbf22a099"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "a0de4e850e5ad1e8fa0c4f48955e8274"
server_key_iv: "2f6b47866fcdc0f6a5661fb2da84cbb5"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.671460 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "50d613922f644c4baf64a1efbf22a099"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "a0de4e850e5ad1e8fa0c4f48955e8274"
server_key_iv: "2f6b47866fcdc0f6a5661fb2da84cbb5"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.676156 19096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:40:11.679654 19111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:11.680805 19096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:11.680959 19096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "50d613922f644c4baf64a1efbf22a099"
format_stamp: "Formatted at 2026-09-26 07:40:11 on dist-test-slave-x492"
server_key: "a0de4e850e5ad1e8fa0c4f48955e8274"
server_key_iv: "2f6b47866fcdc0f6a5661fb2da84cbb5"
server_key_version: "encryptionkey@0"
I20260926 07:40:11.681164 19096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:11.694208 19096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:11.697808 19096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:11.698060 19096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:11.700510 19096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:11.700609 19096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:11.700734 19096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:11.700810 19096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:40:11 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408411, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:11.723320 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.701104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:36893 (local address 127.11.9.126:40473)
0926 07:40:11.701293 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:11.701298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:11.702754 (+ 1456us) server_negotiation.cc:407] Connection header received
0926 07:40:11.704780 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:11.704785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:11.704906 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:11.705019 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:11.706549 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.707635 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.711022 (+ 3387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.711570 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.711712 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.716158 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:11.716163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:11.716166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:11.716197 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:11.719919 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.720822 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.720830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.720835 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.721003 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.721503 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.721510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.721515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.721933 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:11.722093 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:11.722766 (+ 673us) server_negotiation.cc:299] Negotiation successful
0926 07:40:11.722978 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:40:11.724309 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.701807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.66:36893)
0926 07:40:11.702608 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:11.702642 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:11.704516 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:11.705298 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:11.705306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:11.705780 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:11.706386 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.706403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.707884 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.707893 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.710831 (+ 2938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.710852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.711902 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.711912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.712176 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.713006 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:11.713044 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:11.715937 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:11.720071 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.720083 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.720104 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.720661 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.721119 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.721126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.721136 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.721370 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.722079 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:11.722086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:11.722403 (+ 317us) client_negotiation.cc:769] Sending connection context
0926 07:40:11.722653 (+ 250us) client_negotiation.cc:240] Negotiation successful
0926 07:40:11.723159 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":380,"threads_started":1}
I20260926 07:40:11.748558 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.730025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.66:52749)
0926 07:40:11.732697 (+ 2672us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:11.732722 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:11.732928 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:11.733262 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:11.733269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:11.733982 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:11.734429 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.734445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.735409 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.735417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.737495 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:11.737511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.738693 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.738704 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:11.738938 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.739766 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:11.739802 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:11.741336 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:11.744923 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.744930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.744933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.745483 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.746209 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:11.746218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:11.746223 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:11.746429 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:11.747213 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:11.747221 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:11.747501 (+ 280us) client_negotiation.cc:769] Sending connection context
0926 07:40:11.747683 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 07:40:11.748063 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2475,"thread_start_us":2306,"threads_started":1}
I20260926 07:40:11.748624 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:11.730059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:52749 (local address 127.11.9.126:40473)
0926 07:40:11.730270 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:11.730278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:11.732833 (+ 2555us) server_negotiation.cc:407] Connection header received
0926 07:40:11.733044 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:11.733048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:11.733102 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:11.733235 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:11.734581 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.735274 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.737709 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:11.738479 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:11.738642 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:11.741577 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:11.741585 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:11.741593 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:11.741642 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:11.744796 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.745626 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.745633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.745638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.745793 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:11.746584 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:11.746589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:11.746592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:11.747083 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:11.747246 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:11.748102 (+ 856us) server_negotiation.cc:299] Negotiation successful
0926 07:40:11.748306 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:40:11.755065 19096 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:40921
I20260926 07:40:11.755087 19230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:40921 every 8 connection(s)
I20260926 07:40:11.756217 19096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:11.758106 19231 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40473
I20260926 07:40:11.758319 19231 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:11.758903 19231 heartbeater.cc:511] Master 127.11.9.126:40473 requested a full tablet report, sending...
I20260926 07:40:11.760144 18885 ts_manager.cc:194] Registered new tserver with Master: 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921)
I20260926 07:40:11.760870 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19096
I20260926 07:40:11.761027 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:11.761300 18885 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:36893
I20260926 07:40:11.761615 11301 external_mini_cluster.cc:1428] Setting key 8af464af2470fbc2d0266562bf74a85e
WARNING: no policy specified for oryx/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
I20260926 07:40:11.804586 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40473
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:12.124229 19235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:12.124550 19235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:12.124727 19235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:12.132121 19235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:12.132264 19235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:12.132429 19235 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 07:40:12.132571 19235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:12.142558 19235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.9.126:40473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:12.143857 19235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:12.145354 19235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:12.158680 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:12.158658 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:12.159166 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:12.159615 19235 server_base.cc:1034] running on GCE node
I20260926 07:40:12.160233 19235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:12.161301 19235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:12.162554 19235 hybrid_clock.cc:648] HybridClock initialized: now 1790408412162474 us; error 55 us; skew 500 ppm
I20260926 07:40:12.162880 19235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408412, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:12.166042 19235 init.cc:377] Logged in from keytab as oryx/127.11.9.67@KRBTEST.COM (short username oryx)
I20260926 07:40:12.167550 19235 webserver.cc:460] Webserver started at http://127.11.9.67:44767/ using document root <none> and password file <none>
I20260926 07:40:12.168243 19235 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:12.168377 19235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:12.168692 19235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:12.171777 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "b8f053c0d3784121bd705962089634b6"
format_stamp: "Formatted at 2026-09-26 07:40:12 on dist-test-slave-x492"
server_key: "b6195d168ca429e373661165fa0e6b3c"
server_key_iv: "b77e36f18eb532fc77ff567ed2c90362"
server_key_version: "encryptionkey@0"
I20260926 07:40:12.172631 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "b8f053c0d3784121bd705962089634b6"
format_stamp: "Formatted at 2026-09-26 07:40:12 on dist-test-slave-x492"
server_key: "b6195d168ca429e373661165fa0e6b3c"
server_key_iv: "b77e36f18eb532fc77ff567ed2c90362"
server_key_version: "encryptionkey@0"
I20260926 07:40:12.177826 19235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 07:40:12.181337 19250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:12.182745 19235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:40:12.182950 19235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "b8f053c0d3784121bd705962089634b6"
format_stamp: "Formatted at 2026-09-26 07:40:12 on dist-test-slave-x492"
server_key: "b6195d168ca429e373661165fa0e6b3c"
server_key_iv: "b77e36f18eb532fc77ff567ed2c90362"
server_key_version: "encryptionkey@0"
I20260926 07:40:12.183247 19235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:12.228360 19235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:12.232860 19235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:12.233151 19235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:12.236017 19235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:12.236133 19235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:12.236276 19235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:12.236367 19235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408412, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:12.262863 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.236717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:34895 (local address 127.11.9.126:40473)
0926 07:40:12.236942 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.236949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.238682 (+ 1733us) server_negotiation.cc:407] Connection header received
0926 07:40:12.240963 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.240970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.241059 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.241193 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:12.244265 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.244981 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.248635 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.249332 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.249477 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.253305 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:12.253314 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:12.253320 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:12.253364 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:12.258322 (+ 4958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.259571 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.259580 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.259584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.259775 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.260525 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.260531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.260535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.261001 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:12.261206 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.262219 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.262464 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 07:40:12.263976 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.237562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.67:34895)
0926 07:40:12.238513 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.238559 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.240606 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.241527 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.241546 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.242632 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:12.244023 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.244066 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.245157 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.245192 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.248432 (+ 3240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.248450 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.249585 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.249592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.249807 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.250483 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:12.250508 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:12.252949 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:12.258485 (+ 5536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.258497 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.258520 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.259379 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.259975 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.259987 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.259992 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.260352 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.261132 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:12.261142 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:12.261753 (+ 611us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.262098 (+ 345us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.262662 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":424,"threads_started":1}
I20260926 07:40:12.268471 19092 heartbeater.cc:503] Master 127.11.9.126:40473 was elected leader, sending a full tablet report...
I20260926 07:40:12.291780 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.274035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:38191 (local address 127.11.9.126:40473)
0926 07:40:12.274233 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.274240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.275286 (+ 1046us) server_negotiation.cc:407] Connection header received
0926 07:40:12.275475 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.275478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.275550 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.275681 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:12.277077 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.278278 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.280653 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.281475 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.281624 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.284166 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:12.284175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:12.284183 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:12.284244 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:12.288278 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.289342 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.289349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.289354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.289520 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.290050 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.290057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.290061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.290450 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:12.290595 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.291183 (+ 588us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.291403 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:40:12.291917 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.274121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.11.9.67:38191)
0926 07:40:12.275142 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.275168 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.275346 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.275717 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.275722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.276375 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:12.276848 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.276871 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.278452 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.278458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.280466 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.280478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.281715 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.281722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.281890 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.282588 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:12.282609 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:12.283935 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:12.288433 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.288439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.288446 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.289140 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.289680 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.289687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.289693 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.289890 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.290674 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:12.290685 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:12.290937 (+ 252us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.291088 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.291403 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":662,"threads_started":1}
I20260926 07:40:12.293334 19235 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38687
I20260926 07:40:12.293367 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38687 every 8 connection(s)
I20260926 07:40:12.295083 19235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:12.297274 19370 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40473
I20260926 07:40:12.297513 19370 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:12.298133 19370 heartbeater.cc:511] Master 127.11.9.126:40473 requested a full tablet report, sending...
I20260926 07:40:12.299432 18885 ts_manager.cc:194] Registered new tserver with Master: b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:12.300261 18885 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:34895
I20260926 07:40:12.304141 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19235
I20260926 07:40:12.304312 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:12.304723 11301 external_mini_cluster.cc:1428] Setting key 9c33773ca68e03c9594c3b4fd0244116
I20260926 07:40:12.307499 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 07:40:12.354705 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:12.344428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.11.9.126:40473)
0926 07:40:12.344598 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.344602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.344909 (+ 307us) server_negotiation.cc:407] Connection header received
0926 07:40:12.345068 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.345071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.345133 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.345284 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:12.346472 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.347401 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.348555 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.348947 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.349065 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.354499 (+ 5434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32872: BlockingRecv error: recv got EOF from 127.0.0.1:32872 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:12.370502 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.357416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.11.9.126:40473)
0926 07:40:12.357633 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.357638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.357844 (+ 206us) server_negotiation.cc:407] Connection header received
0926 07:40:12.357957 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.357961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.358035 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.358118 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:12.359152 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.359867 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.361044 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.361488 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.361582 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.365004 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:12.365009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:12.365012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:12.365045 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:12.367952 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.368594 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.368601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.368604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.368724 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.369087 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.369093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.369096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.369432 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:12.369713 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.369990 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.370205 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 07:40:12.374306 18885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:12.381084 18885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:12.412436 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.394794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.11.9.65:45277)
0926 07:40:12.404802 (+ 10008us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.404814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.404847 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:40:12.404950 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.404960 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.405241 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.406213 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.406398 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.407911 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.410426 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.411347 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.411504 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.411729 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.411963 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.412122 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9840,"thread_start_us":9722,"threads_started":1}
I20260926 07:40:12.412551 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.394458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45277 (local address 127.0.0.1:52948)
0926 07:40:12.404168 (+ 9710us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.404196 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.404303 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.405481 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.405486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.405515 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.406017 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.406030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.408043 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.408048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.410246 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.410270 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.411550 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.411556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.411724 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.411786 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.411922 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.412114 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9591,"mutex_wait_us":1,"thread_start_us":10861,"threads_started":1}
I20260926 07:40:12.413364 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.392918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:40921 (local address 127.0.0.1:39936)
0926 07:40:12.393791 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.393843 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.394039 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.406759 (+ 12720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.406764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.406788 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.407120 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.407130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.408751 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.408755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.410831 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.410848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.412564 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.412572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.412736 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.412770 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.412829 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.412982 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":51,"thread_start_us":477,"threads_started":1}
I20260926 07:40:12.413908 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.393064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38687 (local address 127.0.0.1:51218)
0926 07:40:12.403848 (+ 10784us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.403874 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.404076 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.405093 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.405105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.405202 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.405847 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.405863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.407621 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.407629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.410207 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.410222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.413124 (+ 2902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.413130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.413306 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.413332 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.413433 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.413553 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10614,"mutex_wait_us":50,"thread_start_us":10483,"threads_started":1}
I20260926 07:40:12.414013 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.393747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.11.9.67:38687)
0926 07:40:12.402733 (+ 8986us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.402745 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.404253 (+ 1508us) server_negotiation.cc:407] Connection header received
0926 07:40:12.404397 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.404408 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.404771 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.405023 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.406135 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.407470 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.410385 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.411815 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.411943 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.412156 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.413491 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.413665 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8771,"thread_start_us":8600,"threads_started":1}
I20260926 07:40:12.414913 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.393063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.11.9.66:40921)
0926 07:40:12.406187 (+ 13124us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.406196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.406222 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:40:12.406311 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.406318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.406540 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.407304 (+ 764us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.407480 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.408594 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.411010 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.412320 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.413975 (+ 1655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.414413 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.414490 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.414641 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12943,"thread_start_us":13543,"threads_started":1}
I20260926 07:40:12.416711 19022 tablet_service.cc:1467] Processing CreateTablet for tablet 19338901923a444e8352768392da8559 (DEFAULT_TABLE table=test-table [id=afa0cfc2088f444e846419bb1e41b7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.417340 19300 tablet_service.cc:1467] Processing CreateTablet for tablet 19338901923a444e8352768392da8559 (DEFAULT_TABLE table=test-table [id=afa0cfc2088f444e846419bb1e41b7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.418278 19022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19338901923a444e8352768392da8559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.418128 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 19338901923a444e8352768392da8559 (DEFAULT_TABLE table=test-table [id=afa0cfc2088f444e846419bb1e41b7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.418711 19300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19338901923a444e8352768392da8559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.419334 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19338901923a444e8352768392da8559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.429567 19394 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Bootstrap starting.
I20260926 07:40:12.432682 19394 tablet_bootstrap.cc:654] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.433597 19395 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Bootstrap starting.
I20260926 07:40:12.434193 19394 log.cc:824] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:12.435132 19396 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Bootstrap starting.
I20260926 07:40:12.437153 19394 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: No bootstrap required, opened a new log
I20260926 07:40:12.437476 19394 ts_tablet_manager.cc:1397] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260926 07:40:12.437896 19395 tablet_bootstrap.cc:654] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.438759 19396 tablet_bootstrap.cc:654] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.439128 19395 log.cc:824] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:12.439973 19396 log.cc:824] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:12.445363 19394 raft_consensus.cc:348] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.445926 19394 raft_consensus.cc:374] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.446070 19394 raft_consensus.cc:729] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b961f4ecf4831a0a4681d60a434a7, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.446755 19394 consensus_queue.cc:260] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.447539 19395 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: No bootstrap required, opened a new log
I20260926 07:40:12.447592 19396 tablet_bootstrap.cc:492] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: No bootstrap required, opened a new log
I20260926 07:40:12.447875 19395 ts_tablet_manager.cc:1397] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260926 07:40:12.447893 19396 ts_tablet_manager.cc:1397] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260926 07:40:12.448500 19394 ts_tablet_manager.cc:1428] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260926 07:40:12.456115 19395 raft_consensus.cc:348] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.456501 19395 raft_consensus.cc:374] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.456597 19395 raft_consensus.cc:729] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f053c0d3784121bd705962089634b6, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.457129 19395 consensus_queue.cc:260] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.457441 19396 raft_consensus.cc:348] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.457796 19396 raft_consensus.cc:374] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.457914 19396 raft_consensus.cc:729] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.458218 19370 heartbeater.cc:503] Master 127.11.9.126:40473 was elected leader, sending a full tablet report...
I20260926 07:40:12.458489 19396 consensus_queue.cc:260] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.458884 19395 ts_tablet_manager.cc:1428] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260926 07:40:12.459749 19231 heartbeater.cc:503] Master 127.11.9.126:40473 was elected leader, sending a full tablet report...
I20260926 07:40:12.460340 19396 ts_tablet_manager.cc:1428] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260926 07:40:12.484511 19401 raft_consensus.cc:482] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:12.484881 19401 raft_consensus.cc:504] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.486593 19401 leader_election.cc:288] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921), fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277)
I20260926 07:40:12.492529 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.487070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:55339 (local address 127.11.9.66:40921)
0926 07:40:12.487245 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.487253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.487344 (+ 91us) server_negotiation.cc:407] Connection header received
0926 07:40:12.487612 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.487618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.487699 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.487804 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.488263 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.489330 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.490888 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.491630 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.491772 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.491871 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.492152 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.492271 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:40:12.492554 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.486936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:40921 (local address 127.11.9.67:55339)
0926 07:40:12.487242 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.487262 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.487428 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.487830 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.487834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.487863 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.488097 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.488107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.489476 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.489480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.490737 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.490748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.491786 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.491791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.491993 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.492023 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.492129 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.492265 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260926 07:40:12.492964 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.487008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:41483 (local address 127.11.9.65:45277)
0926 07:40:12.487265 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.487273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.487990 (+ 717us) server_negotiation.cc:407] Connection header received
0926 07:40:12.488149 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.488153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.488265 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.488335 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.488928 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.489616 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.491288 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.492035 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.492186 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.492287 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.492612 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.492730 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260926 07:40:12.493057 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.487066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45277 (local address 127.11.9.67:41483)
0926 07:40:12.487888 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.487906 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.488021 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.488381 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.488385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.488408 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.488754 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.488764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.489764 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.489769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.491122 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.491134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.492262 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.492269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.492430 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.492456 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.492583 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.492698 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":69,"thread_start_us":409,"threads_started":1}
I20260926 07:40:12.493557 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19338901923a444e8352768392da8559" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d613922f644c4baf64a1efbf22a099" is_pre_election: true
I20260926 07:40:12.494051 19186 raft_consensus.cc:2454] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 0.
I20260926 07:40:12.493975 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19338901923a444e8352768392da8559" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" is_pre_election: true
I20260926 07:40:12.494400 19042 raft_consensus.cc:2454] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 0.
I20260926 07:40:12.494743 19252 leader_election.cc:302] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6; no voters:
I20260926 07:40:12.495146 19401 raft_consensus.cc:2781] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:12.495290 19401 raft_consensus.cc:482] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:12.495405 19401 raft_consensus.cc:3037] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.497627 19401 raft_consensus.cc:504] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.498191 19401 leader_election.cc:288] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 election: Requested vote from peers 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921), fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277)
I20260926 07:40:12.498616 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19338901923a444e8352768392da8559" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d613922f644c4baf64a1efbf22a099"
I20260926 07:40:12.498621 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19338901923a444e8352768392da8559" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
I20260926 07:40:12.498809 19186 raft_consensus.cc:3037] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.498821 19042 raft_consensus.cc:3037] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.500854 19186 raft_consensus.cc:2454] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 1.
I20260926 07:40:12.500859 19042 raft_consensus.cc:2454] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 1.
I20260926 07:40:12.501250 19252 leader_election.cc:302] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6; no voters:
I20260926 07:40:12.501506 19401 raft_consensus.cc:2781] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:12.501940 19401 raft_consensus.cc:686] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 1 LEADER]: Becoming Leader. State: Replica: b8f053c0d3784121bd705962089634b6, State: Running, Role: LEADER
I20260926 07:40:12.502413 19401 consensus_queue.cc:237] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } }
I20260926 07:40:12.506733 18884 catalog_manager.cc:5495] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8f053c0d3784121bd705962089634b6 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260926 07:40:12.508131 19232 tablet.cc:2369] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:40:12.511638 19093 tablet.cc:2369] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:40:12.538426 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.533560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.11.9.67:38687)
0926 07:40:12.533736 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.533741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.533758 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:40:12.533940 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.533943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.534016 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.534135 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:12.534434 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.535104 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.536437 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.536837 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.536927 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.537367 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:12.537901 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:12.538012 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.538074 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.538131 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:40:12.545951 19042 raft_consensus.cc:1261] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Refusing update from remote peer b8f053c0d3784121bd705962089634b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:12.546011 19186 raft_consensus.cc:1261] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Refusing update from remote peer b8f053c0d3784121bd705962089634b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:12.546903 19404 consensus_queue.cc:1035] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:12.547219 19401 consensus_queue.cc:1035] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:40:12.547577 19371 tablet.cc:2369] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:40:12.558871 19409 mvcc.cc:205] Tried to move back new op lower bound from 7333512857779400704 to 7333512857614159872. Current Snapshot: MvccSnapshot[applied={T|T < 7333512857779400704}]
I20260926 07:40:12.576433 18884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:12.576705 18884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:12.581059 18884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Sending DeleteTablet for 3 replicas of tablet 19338901923a444e8352768392da8559
I20260926 07:40:12.582219 19161 tablet_service.cc:1514] Processing DeleteTablet for tablet 19338901923a444e8352768392da8559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:12 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:39936
I20260926 07:40:12.582247 19300 tablet_service.cc:1514] Processing DeleteTablet for tablet 19338901923a444e8352768392da8559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:12 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 07:40:12.582590 19022 tablet_service.cc:1514] Processing DeleteTablet for tablet 19338901923a444e8352768392da8559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:12 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52948
I20260926 07:40:12.584995 18884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:12.585861 18884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:12.594894 19419 tablet_replica.cc:330] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: stopping tablet replica
I20260926 07:40:12.595790 19299 tablet_service.cc:1467] Processing CreateTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f (DEFAULT_TABLE table=test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.596215 19160 tablet_service.cc:1467] Processing CreateTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f (DEFAULT_TABLE table=test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.596473 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ba008c3edb48f1abb7f7c077ea704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.596519 19419 raft_consensus.cc:2229] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:12.596901 19160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ba008c3edb48f1abb7f7c077ea704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.597034 19419 raft_consensus.cc:2258] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:12.600472 19419 ts_tablet_manager.cc:1905] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:12.602320 19021 tablet_service.cc:1467] Processing CreateTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f (DEFAULT_TABLE table=test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:12.602780 19021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ba008c3edb48f1abb7f7c077ea704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.606122 19417 tablet_replica.cc:330] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: stopping tablet replica
I20260926 07:40:12.606534 19417 raft_consensus.cc:2229] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:12.607010 19417 raft_consensus.cc:2258] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:12.609196 19417 ts_tablet_manager.cc:1905] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:12.610659 19396 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Bootstrap starting.
I20260926 07:40:12.611354 19394 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Bootstrap starting.
I20260926 07:40:12.611515 19418 tablet_replica.cc:330] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: stopping tablet replica
I20260926 07:40:12.612392 19396 tablet_bootstrap.cc:654] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.612856 19418 raft_consensus.cc:2229] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:12.612927 19394 tablet_bootstrap.cc:654] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.613485 19418 raft_consensus.cc:2258] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:12.616454 19418 ts_tablet_manager.cc:1905] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:12.617012 19417 ts_tablet_manager.cc:1918] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:12.617187 19417 log.cc:1196] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/19338901923a444e8352768392da8559
I20260926 07:40:12.617431 19395 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Bootstrap starting.
I20260926 07:40:12.617743 19417 ts_tablet_manager.cc:1939] T 19338901923a444e8352768392da8559 P 50d613922f644c4baf64a1efbf22a099: Deleting consensus metadata
I20260926 07:40:12.619516 18870 catalog_manager.cc:4841] TS 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921): tablet 19338901923a444e8352768392da8559 (table test-table [id=afa0cfc2088f444e846419bb1e41b7af]) successfully deleted
I20260926 07:40:12.619582 19395 tablet_bootstrap.cc:654] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.621853 19419 ts_tablet_manager.cc:1918] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:12.622056 19419 log.cc:1196] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/19338901923a444e8352768392da8559
I20260926 07:40:12.622617 19395 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: No bootstrap required, opened a new log
I20260926 07:40:12.622639 19419 ts_tablet_manager.cc:1939] T 19338901923a444e8352768392da8559 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting consensus metadata
I20260926 07:40:12.622817 19395 ts_tablet_manager.cc:1397] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 07:40:12.623800 19395 raft_consensus.cc:348] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.624138 19395 raft_consensus.cc:374] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.624305 19395 raft_consensus.cc:729] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f053c0d3784121bd705962089634b6, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.624322 18869 catalog_manager.cc:4841] TS fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277): tablet 19338901923a444e8352768392da8559 (table test-table [id=afa0cfc2088f444e846419bb1e41b7af]) successfully deleted
I20260926 07:40:12.624679 19395 consensus_queue.cc:260] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.625274 19394 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: No bootstrap required, opened a new log
I20260926 07:40:12.625409 19394 ts_tablet_manager.cc:1397] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260926 07:40:12.626039 19395 ts_tablet_manager.cc:1428] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 07:40:12.626236 19394 raft_consensus.cc:348] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.626535 19394 raft_consensus.cc:374] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.626649 19394 raft_consensus.cc:729] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b961f4ecf4831a0a4681d60a434a7, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.626950 19394 consensus_queue.cc:260] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.627604 19394 ts_tablet_manager.cc:1428] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:12.630991 19396 tablet_bootstrap.cc:492] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: No bootstrap required, opened a new log
I20260926 07:40:12.631204 19396 ts_tablet_manager.cc:1397] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.002s
I20260926 07:40:12.632007 19396 raft_consensus.cc:348] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.632407 19396 raft_consensus.cc:374] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.632513 19396 raft_consensus.cc:729] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.632752 19396 consensus_queue.cc:260] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.633442 19396 ts_tablet_manager.cc:1428] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 07:40:12.641628 19418 ts_tablet_manager.cc:1918] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:12.641821 19418 log.cc:1196] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/19338901923a444e8352768392da8559
I20260926 07:40:12.642388 19418 ts_tablet_manager.cc:1939] T 19338901923a444e8352768392da8559 P b8f053c0d3784121bd705962089634b6: Deleting consensus metadata
I20260926 07:40:12.644110 18871 catalog_manager.cc:4841] TS b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687): tablet 19338901923a444e8352768392da8559 (table test-table [id=afa0cfc2088f444e846419bb1e41b7af]) successfully deleted
I20260926 07:40:12.654207 19402 raft_consensus.cc:482] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:12.654443 19402 raft_consensus.cc:504] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.656065 19402 leader_election.cc:288] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:12.664780 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.656637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:47857 (local address 127.11.9.65:45277)
0926 07:40:12.656811 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.656817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.657826 (+ 1009us) server_negotiation.cc:407] Connection header received
0926 07:40:12.658112 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.658119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.658195 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.658241 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.659131 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.659862 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.662542 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.663742 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.663870 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.663967 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.664372 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.664513 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:40:12.664813 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.656938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:52961 (local address 127.11.9.67:38687)
0926 07:40:12.657119 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.657125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.658033 (+ 908us) server_negotiation.cc:407] Connection header received
0926 07:40:12.658146 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.658152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.658232 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.658328 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:12.659017 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.660174 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.662540 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.663761 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.663869 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.663970 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.664434 (+ 464us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.664549 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:40:12.664850 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.656286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45277 (local address 127.11.9.66:47857)
0926 07:40:12.657688 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.657713 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.657945 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.658462 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.658468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.658506 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.658957 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.658971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.660002 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.660010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.662349 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.662364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.663948 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.663955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.664178 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.664226 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.664330 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.664467 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":25,"thread_start_us":1376,"threads_started":1}
I20260926 07:40:12.665547 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ba008c3edb48f1abb7f7c077ea704f" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" is_pre_election: true
I20260926 07:40:12.665643 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.656510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38687 (local address 127.11.9.66:52961)
0926 07:40:12.657911 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.657928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.658038 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.658372 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.658379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.658421 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:12.658836 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.658850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.660311 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.660321 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.662356 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.662364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.664110 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.664114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.664244 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.664266 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.665153 (+ 887us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.665296 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":7,"thread_start_us":848,"threads_started":1}
I20260926 07:40:12.665791 19042 raft_consensus.cc:2454] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:12.666277 19112 leader_election.cc:302] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, fe3b961f4ecf4831a0a4681d60a434a7; no voters:
I20260926 07:40:12.666677 19402 raft_consensus.cc:2781] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:12.666491 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ba008c3edb48f1abb7f7c077ea704f" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6" is_pre_election: true
I20260926 07:40:12.666802 19402 raft_consensus.cc:482] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:12.666884 19402 raft_consensus.cc:3037] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.666954 19325 raft_consensus.cc:2454] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:12.669157 19402 raft_consensus.cc:504] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.669834 19402 leader_election.cc:288] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Requested vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:12.670164 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ba008c3edb48f1abb7f7c077ea704f" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
I20260926 07:40:12.670378 19042 raft_consensus.cc:3037] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.670513 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ba008c3edb48f1abb7f7c077ea704f" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6"
I20260926 07:40:12.670727 19325 raft_consensus.cc:3037] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:12.672636 19042 raft_consensus.cc:2454] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:12.672994 19112 leader_election.cc:302] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, fe3b961f4ecf4831a0a4681d60a434a7; no voters:
I20260926 07:40:12.673291 19402 raft_consensus.cc:2781] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:12.673796 19402 raft_consensus.cc:686] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Becoming Leader. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Running, Role: LEADER
I20260926 07:40:12.674010 19325 raft_consensus.cc:2454] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:12.674255 19402 consensus_queue.cc:237] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.682785 18884 catalog_manager.cc:5495] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d613922f644c4baf64a1efbf22a099 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "50d613922f644c4baf64a1efbf22a099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } health_report { overall_health: UNKNOWN } } }
Sep 26 07:40:12 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408410, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.126@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:12.730592 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.715678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40473 (local address 127.0.0.1:32890)
0926 07:40:12.716480 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:12.716500 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:12.716627 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:12.716980 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:12.716983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:12.717721 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:12.717986 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.717995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.719441 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.719444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.720353 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:12.720362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.721314 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.721318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:12.721426 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.722129 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:12.722145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:12.724397 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:12.728199 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.728208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.728214 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.728649 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.729026 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:12.729029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:12.729031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:12.729157 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:12.729824 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:12.729830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:12.729966 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 07:40:12.730099 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 07:40:12.730279 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":515,"threads_started":1}
I20260926 07:40:12.730983 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:12.715731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.11.9.126:40473)
0926 07:40:12.715995 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:12.716004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:12.716603 (+ 599us) server_negotiation.cc:407] Connection header received
0926 07:40:12.716740 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:12.716746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:12.716826 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:12.716944 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:12.718139 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.719322 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.720498 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:12.721136 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:12.721281 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:12.724605 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:12.724612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:12.724619 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:12.724655 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:12.728066 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.728767 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.728773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.728777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.728915 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:12.729333 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:12.729338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:12.729341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:12.729696 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:12.729843 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:12.730223 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 07:40:12.730432 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260926 07:40:12.735749 18884 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:32890:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 07:40:12.736788 18884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:12.743610 19160 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a82219171774bf79c3df6cb717cfe68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:40:12.743549 19021 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a82219171774bf79c3df6cb717cfe68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:40:12.744091 19160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.744100 19021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.745723 19299 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a82219171774bf79c3df6cb717cfe68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 07:40:12.746202 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3e5e7be3df4eab97d05ba20ce5c09d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:12.750716 19396 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: Bootstrap starting.
I20260926 07:40:12.751516 19394 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: Bootstrap starting.
I20260926 07:40:12.752360 19396 tablet_bootstrap.cc:654] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.753154 19394 tablet_bootstrap.cc:654] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.754689 19395 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: Bootstrap starting.
I20260926 07:40:12.756461 19395 tablet_bootstrap.cc:654] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:12.763916 19394 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: No bootstrap required, opened a new log
I20260926 07:40:12.764142 19394 ts_tablet_manager.cc:1397] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260926 07:40:12.764927 19394 raft_consensus.cc:348] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.765277 19394 raft_consensus.cc:374] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.765403 19394 raft_consensus.cc:729] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b961f4ecf4831a0a4681d60a434a7, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.765738 19394 consensus_queue.cc:260] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.766588 19394 ts_tablet_manager.cc:1428] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 07:40:12.766413 19400 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:12.766925 19400 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:12.778592 19395 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: No bootstrap required, opened a new log
I20260926 07:40:12.778800 19395 ts_tablet_manager.cc:1397] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.004s
I20260926 07:40:12.779443 19396 tablet_bootstrap.cc:492] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: No bootstrap required, opened a new log
I20260926 07:40:12.779661 19396 ts_tablet_manager.cc:1397] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.011s
I20260926 07:40:12.779739 19395 raft_consensus.cc:348] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.780547 19396 raft_consensus.cc:348] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.780841 19396 raft_consensus.cc:374] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.780027 19395 raft_consensus.cc:374] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:12.780990 19396 raft_consensus.cc:729] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.781123 19395 raft_consensus.cc:729] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f053c0d3784121bd705962089634b6, State: Initialized, Role: FOLLOWER
I20260926 07:40:12.781426 19395 consensus_queue.cc:260] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.781426 19396 consensus_queue.cc:260] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:12.781888 19401 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:12.781877 19402 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:12.782135 19401 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:12.782151 19402 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:12.783259 19395 ts_tablet_manager.cc:1428] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 07:40:12.785806 19396 ts_tablet_manager.cc:1428] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260926 07:40:13.044485 19401 raft_consensus.cc:482] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:13.044697 19401 raft_consensus.cc:504] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.045399 19401 leader_election.cc:288] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921)
I20260926 07:40:13.046031 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3e5e7be3df4eab97d05ba20ce5c09d" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d613922f644c4baf64a1efbf22a099" is_pre_election: true
I20260926 07:40:13.046016 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3e5e7be3df4eab97d05ba20ce5c09d" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" is_pre_election: true
I20260926 07:40:13.046276 19186 raft_consensus.cc:2454] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 0.
I20260926 07:40:13.046294 19042 raft_consensus.cc:2454] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 0.
I20260926 07:40:13.046728 19252 leader_election.cc:302] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6, fe3b961f4ecf4831a0a4681d60a434a7; no voters:
I20260926 07:40:13.047013 19401 raft_consensus.cc:2781] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:13.047139 19401 raft_consensus.cc:482] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:13.047246 19401 raft_consensus.cc:3037] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.049595 19401 raft_consensus.cc:504] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.050091 19401 leader_election.cc:288] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 election: Requested vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921)
I20260926 07:40:13.050567 19042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3e5e7be3df4eab97d05ba20ce5c09d" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
I20260926 07:40:13.050587 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3e5e7be3df4eab97d05ba20ce5c09d" candidate_uuid: "b8f053c0d3784121bd705962089634b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50d613922f644c4baf64a1efbf22a099"
I20260926 07:40:13.050783 19186 raft_consensus.cc:3037] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.050804 19042 raft_consensus.cc:3037] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.053646 19042 raft_consensus.cc:2454] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 1.
I20260926 07:40:13.053664 19186 raft_consensus.cc:2454] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f053c0d3784121bd705962089634b6 in term 1.
I20260926 07:40:13.054054 19252 leader_election.cc:302] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6; no voters:
I20260926 07:40:13.054340 19401 raft_consensus.cc:2781] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:13.054483 19401 raft_consensus.cc:686] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [term 1 LEADER]: Becoming Leader. State: Replica: b8f053c0d3784121bd705962089634b6, State: Running, Role: LEADER
I20260926 07:40:13.054736 19401 consensus_queue.cc:237] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.056257 19404 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: TxnStatusTablet state changed. Reason: New leader b8f053c0d3784121bd705962089634b6. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.056514 19404 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:40:13.057125 19437 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 07:40:13.057418 19437 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 07:40:13.057843 18885 catalog_manager.cc:5495] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8f053c0d3784121bd705962089634b6 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } health_report { overall_health: HEALTHY } } }
I20260926 07:40:13.079818 19402 consensus_queue.cc:1035] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.086843 19402 consensus_queue.cc:1035] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.132323 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.126977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38687 (local address 127.0.0.1:51230)
0926 07:40:13.127197 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:13.127218 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:13.127374 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:13.127787 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:13.127790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:13.127800 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:40:13.127954 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.127961 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.129332 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.129336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.130326 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.130336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.131288 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.131292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.131420 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.131502 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.131918 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.131932 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 07:40:13.131991 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 07:40:13.132104 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 07:40:13.132378 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.127122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.11.9.67:38687)
0926 07:40:13.127321 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.127328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.127348 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:40:13.127547 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.127555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.127659 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.127756 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:13.128076 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.129219 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.130460 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.131141 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.131269 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.131689 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.131821 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.131921 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.132014 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.132093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:40:13.135985 19186 raft_consensus.cc:1261] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Refusing update from remote peer b8f053c0d3784121bd705962089634b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:13.136056 19042 raft_consensus.cc:1261] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Refusing update from remote peer b8f053c0d3784121bd705962089634b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:13.136657 19404 consensus_queue.cc:1035] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.136894 19401 consensus_queue.cc:1035] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.139724 19425 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: TxnStatusTablet state changed. Reason: New leader b8f053c0d3784121bd705962089634b6. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.140007 19425 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:13.141450 19400 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: TxnStatusTablet state changed. Reason: New leader b8f053c0d3784121bd705962089634b6. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.141722 19400 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:13.142364 19443 mvcc.cc:205] Tried to move back new op lower bound from 7333512860199313408 to 7333512859874332672. Current Snapshot: MvccSnapshot[applied={T|T < 7333512860199313408}]
I20260926 07:40:13.144155 19440 mvcc.cc:205] Tried to move back new op lower bound from 7333512860199313408 to 7333512859874332672. Current Snapshot: MvccSnapshot[applied={T|T < 7333512860199313408}]
I20260926 07:40:13.151839 19425 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.152077 19425 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P 50d613922f644c4baf64a1efbf22a099: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:13.153513 19401 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.153823 19401 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:40:13.155682 19400 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.155905 19400 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P fe3b961f4ecf4831a0a4681d60a434a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 07:40:13.157073 19404 tablet_replica.cc:439] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8f053c0d3784121bd705962089634b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } } }
I20260926 07:40:13.157354 19404 tablet_replica.cc:442] T 1f3e5e7be3df4eab97d05ba20ce5c09d P b8f053c0d3784121bd705962089634b6: This TxnStatusTablet replica's current role is: LEADER
I20260926 07:40:13.168955 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.162148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.11.9.66:40921)
0926 07:40:13.163438 (+ 1290us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.163442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.163456 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:40:13.163537 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.163540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.163613 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.163738 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:13.164229 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.164968 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.166744 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.167245 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.167373 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.167839 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.168359 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.168513 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.168609 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.168682 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":1083,"threads_started":1}
I20260926 07:40:13.184270 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.178392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38687 (local address 127.11.9.66:55553)
0926 07:40:13.178885 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:13.178900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:13.179003 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:13.179324 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:13.179326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:13.179335 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:40:13.179507 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.179513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.180900 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.180903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.181978 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.181989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.182992 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.182996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.183115 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.183162 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.183733 (+ 571us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.183750 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 07:40:13.183891 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 07:40:13.184005 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":335,"threads_started":1}
I20260926 07:40:13.184334 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.178500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:55553 (local address 127.11.9.67:38687)
0926 07:40:13.178714 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.178720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.178985 (+ 265us) server_negotiation.cc:407] Connection header received
0926 07:40:13.179128 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.179134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.179213 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.179312 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:13.179648 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.180784 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.182110 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.182843 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.182985 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.183321 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.183490 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.183576 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.183932 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.184016 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:40:13.222360 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.215073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:56563 (local address 127.11.9.66:40921)
0926 07:40:13.215254 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.215261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.215913 (+ 652us) server_negotiation.cc:407] Connection header received
0926 07:40:13.216120 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.216125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.216205 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.216290 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:13.217050 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.218212 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.220416 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.221052 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.221144 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.221698 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.221820 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.221910 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.222007 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.222089 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 07:40:13.222380 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.215138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:40921 (local address 127.11.9.67:56563)
0926 07:40:13.215811 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:13.215827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:13.215983 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:13.216457 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:13.216461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:13.216477 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 07:40:13.216838 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.216857 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.218382 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.218387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.220255 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:13.220269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.221350 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.221354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:13.221477 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.221524 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.221916 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.221924 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 07:40:13.221981 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 07:40:13.222081 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":347,"threads_started":1}
I20260926 07:40:13.263311 18885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:13.263553 18885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:13.267462 18885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Sending DeleteTablet for 3 replicas of tablet 63ba008c3edb48f1abb7f7c077ea704f
I20260926 07:40:13.269337 19021 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52948
I20260926 07:40:13.270056 19160 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:39936
I20260926 07:40:13.270499 19299 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ba008c3edb48f1abb7f7c077ea704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 07:40:13.287703 19462 tablet_replica.cc:330] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: stopping tablet replica
I20260926 07:40:13.288650 19462 raft_consensus.cc:2229] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:13.293267 19462 raft_consensus.cc:2258] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.298231 19462 ts_tablet_manager.cc:1905] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:13.320585 19462 ts_tablet_manager.cc:1918] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:13.320735 19462 log.cc:1196] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/63ba008c3edb48f1abb7f7c077ea704f
I20260926 07:40:13.321194 19462 ts_tablet_manager.cc:1939] T 63ba008c3edb48f1abb7f7c077ea704f P 50d613922f644c4baf64a1efbf22a099: Deleting consensus metadata
I20260926 07:40:13.322405 18870 catalog_manager.cc:4841] TS 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921): tablet 63ba008c3edb48f1abb7f7c077ea704f (table test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]) successfully deleted
I20260926 07:40:13.327361 19463 tablet_replica.cc:330] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: stopping tablet replica
I20260926 07:40:13.327598 19463 raft_consensus.cc:2229] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:13.327924 19463 raft_consensus.cc:2258] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.328603 19461 tablet_replica.cc:330] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: stopping tablet replica
I20260926 07:40:13.328809 19461 raft_consensus.cc:2229] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:13.329059 19461 raft_consensus.cc:2258] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.330262 19463 ts_tablet_manager.cc:1905] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 07:40:13 dist-test-slave-x492 krb5kdc[18840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260926 07:40:13.331982 19461 ts_tablet_manager.cc:1905] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:13.347148 19461 ts_tablet_manager.cc:1918] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:13.347296 19461 log.cc:1196] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/63ba008c3edb48f1abb7f7c077ea704f
I20260926 07:40:13.347725 19461 ts_tablet_manager.cc:1939] T 63ba008c3edb48f1abb7f7c077ea704f P fe3b961f4ecf4831a0a4681d60a434a7: Deleting consensus metadata
I20260926 07:40:13.348822 18869 catalog_manager.cc:4841] TS fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277): tablet 63ba008c3edb48f1abb7f7c077ea704f (table test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]) successfully deleted
I20260926 07:40:13.352272 19463 ts_tablet_manager.cc:1918] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:13.352435 19463 log.cc:1196] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/63ba008c3edb48f1abb7f7c077ea704f
I20260926 07:40:13.352942 19463 ts_tablet_manager.cc:1939] T 63ba008c3edb48f1abb7f7c077ea704f P b8f053c0d3784121bd705962089634b6: Deleting consensus metadata
I20260926 07:40:13.354120 18871 catalog_manager.cc:4841] TS b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687): tablet 63ba008c3edb48f1abb7f7c077ea704f (table test-table [id=9e8f968d3c8f4966ad22c16d7b8943b1]) successfully deleted
Sep 26 07:40:13 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:13 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 07:40:13.393836 19470 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:13.370512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.11.9.126:40473)
0926 07:40:13.377610 (+ 7098us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.377620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.380865 (+ 3245us) server_negotiation.cc:407] Connection header received
0926 07:40:13.381102 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.381108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.381219 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.381391 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:13.383174 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.384389 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.386484 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.387126 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.387240 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.393518 (+ 6278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32900: BlockingRecv error: recv got EOF from 127.0.0.1:32900 (error 108)
Metrics: {"server-negotiator.queue_time_us":6914,"thread_start_us":6761,"threads_started":1}
Sep 26 07:40:13 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM
I20260926 07:40:13.416132 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.397262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.11.9.126:40473)
0926 07:40:13.397465 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.397471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.397837 (+ 366us) server_negotiation.cc:407] Connection header received
0926 07:40:13.398035 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.398040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.398126 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.398234 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:13.399812 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.400881 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.403297 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.403922 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.404078 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.409421 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:13.409429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:13.409435 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:13.409482 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:13.413064 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:13.413970 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:13.413976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:13.413980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:13.414137 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:13.414578 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:13.414583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:13.414587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:13.414938 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:13.415188 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.415538 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.415769 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 07:40:13.420296 18885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:13.421043 18885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:13.428846 19299 tablet_service.cc:1467] Processing CreateTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 (DEFAULT_TABLE table=test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.428840 19160 tablet_service.cc:1467] Processing CreateTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 (DEFAULT_TABLE table=test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.429376 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879018fb2ad7402ebc5c1596595a0b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.429394 19160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879018fb2ad7402ebc5c1596595a0b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.434336 19021 tablet_service.cc:1467] Processing CreateTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 (DEFAULT_TABLE table=test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.434784 19021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879018fb2ad7402ebc5c1596595a0b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.455103 19479 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Bootstrap starting.
I20260926 07:40:13.457661 19478 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Bootstrap starting.
I20260926 07:40:13.459775 19478 tablet_bootstrap.cc:654] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.460285 19479 tablet_bootstrap.cc:654] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.463972 19479 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: No bootstrap required, opened a new log
I20260926 07:40:13.464192 19479 ts_tablet_manager.cc:1397] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260926 07:40:13.465019 19479 raft_consensus.cc:348] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.465391 19479 raft_consensus.cc:374] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.465525 19479 raft_consensus.cc:729] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b961f4ecf4831a0a4681d60a434a7, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.465842 19479 consensus_queue.cc:260] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.466627 19477 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Bootstrap starting.
I20260926 07:40:13.467697 19479 ts_tablet_manager.cc:1428] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 07:40:13.469823 19477 tablet_bootstrap.cc:654] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.472250 19478 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: No bootstrap required, opened a new log
I20260926 07:40:13.472401 19478 ts_tablet_manager.cc:1397] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260926 07:40:13.473284 19478 raft_consensus.cc:348] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.473546 19478 raft_consensus.cc:374] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.473649 19478 raft_consensus.cc:729] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f053c0d3784121bd705962089634b6, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.473902 19478 consensus_queue.cc:260] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.474622 19478 ts_tablet_manager.cc:1428] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 07:40:13.482673 19477 tablet_bootstrap.cc:492] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: No bootstrap required, opened a new log
I20260926 07:40:13.482903 19477 ts_tablet_manager.cc:1397] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.010s
I20260926 07:40:13.483724 19477 raft_consensus.cc:348] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.484028 19477 raft_consensus.cc:374] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.484125 19477 raft_consensus.cc:729] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.484364 19477 consensus_queue.cc:260] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.485488 19477 ts_tablet_manager.cc:1428] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:13.819008 19425 raft_consensus.cc:482] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:13.819216 19425 raft_consensus.cc:504] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.819998 19425 leader_election.cc:288] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:13.820526 19041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879018fb2ad7402ebc5c1596595a0b88" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" is_pre_election: true
I20260926 07:40:13.820590 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879018fb2ad7402ebc5c1596595a0b88" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6" is_pre_election: true
I20260926 07:40:13.820789 19041 raft_consensus.cc:2454] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:13.820962 19325 raft_consensus.cc:2454] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:13.821314 19112 leader_election.cc:302] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, fe3b961f4ecf4831a0a4681d60a434a7; no voters:
I20260926 07:40:13.821568 19425 raft_consensus.cc:2781] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:13.821682 19425 raft_consensus.cc:482] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:13.821774 19425 raft_consensus.cc:3037] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.823695 19425 raft_consensus.cc:504] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.824239 19425 leader_election.cc:288] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Requested vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:13.824630 19041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879018fb2ad7402ebc5c1596595a0b88" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
I20260926 07:40:13.824698 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879018fb2ad7402ebc5c1596595a0b88" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6"
I20260926 07:40:13.824817 19041 raft_consensus.cc:3037] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.824887 19325 raft_consensus.cc:3037] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:13.826890 19041 raft_consensus.cc:2454] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:13.826938 19325 raft_consensus.cc:2454] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:13.827338 19114 leader_election.cc:302] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6; no voters:
I20260926 07:40:13.827613 19425 raft_consensus.cc:2781] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:13.827991 19425 raft_consensus.cc:686] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Becoming Leader. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Running, Role: LEADER
I20260926 07:40:13.828255 19425 consensus_queue.cc:237] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.830753 18885 catalog_manager.cc:5495] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d613922f644c4baf64a1efbf22a099 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "50d613922f644c4baf64a1efbf22a099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } health_report { overall_health: UNKNOWN } } }
I20260926 07:40:13.904757 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:13.900202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.11.9.66:40921)
0926 07:40:13.900687 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:13.900691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:13.900706 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:40:13.900764 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:13.900767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:13.900846 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:13.900931 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:13.901292 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.902120 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.903352 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:13.903733 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:13.903828 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:13.904187 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:13.904276 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:13.904341 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:13.904467 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 07:40:13.904525 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":278,"threads_started":1}
I20260926 07:40:13.907802 19041 raft_consensus.cc:1261] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Refusing update from remote peer 50d613922f644c4baf64a1efbf22a099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:13.907816 19325 raft_consensus.cc:1261] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Refusing update from remote peer 50d613922f644c4baf64a1efbf22a099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:13.908411 19425 consensus_queue.cc:1035] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.908718 19483 consensus_queue.cc:1035] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:13.922909 18885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:13.923221 18885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:13.926555 18885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Sending DeleteTablet for 3 replicas of tablet 879018fb2ad7402ebc5c1596595a0b88
I20260926 07:40:13.927338 19021 tablet_service.cc:1514] Processing DeleteTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52948
I20260926 07:40:13.927618 19299 tablet_service.cc:1514] Processing DeleteTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 07:40:13.927618 19160 tablet_service.cc:1514] Processing DeleteTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:39936
I20260926 07:40:13.928046 19300 tablet_service.cc:1514] Processing DeleteTablet for tablet 879018fb2ad7402ebc5c1596595a0b88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:13 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 07:40:13.929513 18885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:13.930575 18885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:13.932790 19497 tablet_replica.cc:330] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: stopping tablet replica
I20260926 07:40:13.932950 19496 tablet_replica.cc:330] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: stopping tablet replica
I20260926 07:40:13.932986 19497 raft_consensus.cc:2229] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:13.933204 19498 tablet_replica.cc:330] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: stopping tablet replica
I20260926 07:40:13.933193 19496 raft_consensus.cc:2229] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:13.933344 19497 raft_consensus.cc:2258] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.933553 19498 raft_consensus.cc:2229] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:13.933698 19496 raft_consensus.cc:2258] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.933951 19498 raft_consensus.cc:2258] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:13.934633 19497 ts_tablet_manager.cc:1905] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:13.934847 19496 ts_tablet_manager.cc:1905] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:13.935257 19498 ts_tablet_manager.cc:1905] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:13.939137 19299 tablet_service.cc:1467] Processing CreateTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe (DEFAULT_TABLE table=test-table [id=0da845308d3648f99064f3dfa1fd871d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.939172 19160 tablet_service.cc:1467] Processing CreateTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe (DEFAULT_TABLE table=test-table [id=0da845308d3648f99064f3dfa1fd871d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.939235 19021 tablet_service.cc:1467] Processing CreateTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe (DEFAULT_TABLE table=test-table [id=0da845308d3648f99064f3dfa1fd871d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:13.939568 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f2c2d6f0ce48a7b7217802db6135fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.939579 19160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f2c2d6f0ce48a7b7217802db6135fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.939589 19021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f2c2d6f0ce48a7b7217802db6135fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:13.941919 19497 ts_tablet_manager.cc:1918] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:13.942032 19497 log.cc:1196] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/879018fb2ad7402ebc5c1596595a0b88
I20260926 07:40:13.942607 19497 ts_tablet_manager.cc:1939] T 879018fb2ad7402ebc5c1596595a0b88 P 50d613922f644c4baf64a1efbf22a099: Deleting consensus metadata
I20260926 07:40:13.943662 18870 catalog_manager.cc:4841] TS 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921): tablet 879018fb2ad7402ebc5c1596595a0b88 (table test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]) successfully deleted
I20260926 07:40:13.945470 19479 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Bootstrap starting.
I20260926 07:40:13.947028 19478 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Bootstrap starting.
I20260926 07:40:13.947140 19479 tablet_bootstrap.cc:654] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.948096 19477 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Bootstrap starting.
I20260926 07:40:13.948767 19498 ts_tablet_manager.cc:1918] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:13.948854 19478 tablet_bootstrap.cc:654] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.948904 19498 log.cc:1196] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/879018fb2ad7402ebc5c1596595a0b88
I20260926 07:40:13.949397 19498 ts_tablet_manager.cc:1939] T 879018fb2ad7402ebc5c1596595a0b88 P b8f053c0d3784121bd705962089634b6: Deleting consensus metadata
I20260926 07:40:13.949787 19477 tablet_bootstrap.cc:654] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:13.950682 18871 catalog_manager.cc:4841] TS b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687): tablet 879018fb2ad7402ebc5c1596595a0b88 (table test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]) successfully deleted
W20260926 07:40:13.951120 18871 catalog_manager.cc:4808] TS b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687): delete failed for tablet 879018fb2ad7402ebc5c1596595a0b88 because the tablet was not found. No further retry: Not found: Tablet not found: 879018fb2ad7402ebc5c1596595a0b88
I20260926 07:40:13.954670 19477 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: No bootstrap required, opened a new log
I20260926 07:40:13.954846 19477 ts_tablet_manager.cc:1397] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260926 07:40:13.955546 19477 raft_consensus.cc:348] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.955667 19496 ts_tablet_manager.cc:1918] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:13.955744 19477 raft_consensus.cc:374] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.955794 19477 raft_consensus.cc:729] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.955793 19496 log.cc:1196] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/879018fb2ad7402ebc5c1596595a0b88
I20260926 07:40:13.956180 19477 consensus_queue.cc:260] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.956274 19496 ts_tablet_manager.cc:1939] T 879018fb2ad7402ebc5c1596595a0b88 P fe3b961f4ecf4831a0a4681d60a434a7: Deleting consensus metadata
I20260926 07:40:13.957396 19477 ts_tablet_manager.cc:1428] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 07:40:13.958884 19478 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: No bootstrap required, opened a new log
I20260926 07:40:13.959023 18869 catalog_manager.cc:4841] TS fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277): tablet 879018fb2ad7402ebc5c1596595a0b88 (table test-table [id=0229d65fd68d444f8ae5fe4f5a41f89b]) successfully deleted
I20260926 07:40:13.959105 19478 ts_tablet_manager.cc:1397] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260926 07:40:13.959933 19478 raft_consensus.cc:348] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.960276 19478 raft_consensus.cc:374] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.960405 19478 raft_consensus.cc:729] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f053c0d3784121bd705962089634b6, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.960690 19478 consensus_queue.cc:260] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.961275 19478 ts_tablet_manager.cc:1428] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 07:40:13.962742 19479 tablet_bootstrap.cc:492] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: No bootstrap required, opened a new log
I20260926 07:40:13.962950 19479 ts_tablet_manager.cc:1397] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.006s
I20260926 07:40:13.963523 19479 raft_consensus.cc:348] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.963785 19479 raft_consensus.cc:374] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:13.963882 19479 raft_consensus.cc:729] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b961f4ecf4831a0a4681d60a434a7, State: Initialized, Role: FOLLOWER
I20260926 07:40:13.964110 19479 consensus_queue.cc:260] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:13.964715 19479 ts_tablet_manager.cc:1428] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260926 07:40:14.087532 19483 raft_consensus.cc:482] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:14.087767 19483 raft_consensus.cc:504] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:14.088464 19483 leader_election.cc:288] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:14.088984 19041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f2c2d6f0ce48a7b7217802db6135fe" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" is_pre_election: true
I20260926 07:40:14.089017 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f2c2d6f0ce48a7b7217802db6135fe" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6" is_pre_election: true
I20260926 07:40:14.089231 19041 raft_consensus.cc:2454] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:14.089365 19325 raft_consensus.cc:2454] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 0.
I20260926 07:40:14.089723 19112 leader_election.cc:302] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, fe3b961f4ecf4831a0a4681d60a434a7; no voters:
I20260926 07:40:14.089983 19483 raft_consensus.cc:2781] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:14.090063 19483 raft_consensus.cc:482] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:14.090140 19483 raft_consensus.cc:3037] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:14.092038 19483 raft_consensus.cc:504] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:14.092542 19483 leader_election.cc:288] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Requested vote from peers fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277), b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687)
I20260926 07:40:14.092954 19325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f2c2d6f0ce48a7b7217802db6135fe" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f053c0d3784121bd705962089634b6"
I20260926 07:40:14.093029 19041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f2c2d6f0ce48a7b7217802db6135fe" candidate_uuid: "50d613922f644c4baf64a1efbf22a099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3b961f4ecf4831a0a4681d60a434a7"
I20260926 07:40:14.093241 19041 raft_consensus.cc:3037] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:14.093235 19325 raft_consensus.cc:3037] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:14.096181 19325 raft_consensus.cc:2454] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:14.096211 19041 raft_consensus.cc:2454] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d613922f644c4baf64a1efbf22a099 in term 1.
I20260926 07:40:14.096570 19114 leader_election.cc:302] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d613922f644c4baf64a1efbf22a099, b8f053c0d3784121bd705962089634b6; no voters:
I20260926 07:40:14.096845 19483 raft_consensus.cc:2781] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:14.097026 19483 raft_consensus.cc:686] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Becoming Leader. State: Replica: 50d613922f644c4baf64a1efbf22a099, State: Running, Role: LEADER
I20260926 07:40:14.097314 19483 consensus_queue.cc:237] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } }
I20260926 07:40:14.099932 18884 catalog_manager.cc:5495] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d613922f644c4baf64a1efbf22a099 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "50d613922f644c4baf64a1efbf22a099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50d613922f644c4baf64a1efbf22a099" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 } health_report { overall_health: UNKNOWN } } }
I20260926 07:40:14.155803 19325 raft_consensus.cc:1261] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Refusing update from remote peer 50d613922f644c4baf64a1efbf22a099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:14.155781 19042 raft_consensus.cc:1261] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Refusing update from remote peer 50d613922f644c4baf64a1efbf22a099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:14.156418 19483 consensus_queue.cc:1035] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3b961f4ecf4831a0a4681d60a434a7" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:14.156689 19492 consensus_queue.cc:1035] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f053c0d3784121bd705962089634b6" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:14.200654 18885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:14.200913 18885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:14.203518 18885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b5eaea124ca4b7bbf5cfc0d74abbe8c: Sending DeleteTablet for 3 replicas of tablet d8f2c2d6f0ce48a7b7217802db6135fe
I20260926 07:40:14.204226 19021 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:14 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:52948
I20260926 07:40:14.204492 19496 tablet_replica.cc:330] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: stopping tablet replica
I20260926 07:40:14.204438 19160 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:14 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:39936
I20260926 07:40:14.204449 19299 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f2c2d6f0ce48a7b7217802db6135fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:14 UTC) from {username='oryx', principal='oryx/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 07:40:14.204695 19496 raft_consensus.cc:2229] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:14.204869 19496 raft_consensus.cc:2258] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:14.205207 19497 tablet_replica.cc:330] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: stopping tablet replica
I20260926 07:40:14.205395 19497 raft_consensus.cc:2229] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:14.205655 19497 raft_consensus.cc:2258] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:14.205928 19496 ts_tablet_manager.cc:1905] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:14.206133 19498 tablet_replica.cc:330] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: stopping tablet replica
I20260926 07:40:14.206334 19498 raft_consensus.cc:2229] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:14.206456 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18956
I20260926 07:40:14.206512 19498 raft_consensus.cc:2258] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:14.207286 19497 ts_tablet_manager.cc:1905] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:14.207504 19498 ts_tablet_manager.cc:1905] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:14.213790 19498 ts_tablet_manager.cc:1918] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:14.213812 19497 ts_tablet_manager.cc:1918] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:14.213932 19498 log.cc:1196] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/d8f2c2d6f0ce48a7b7217802db6135fe
I20260926 07:40:14.213940 19497 log.cc:1196] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/d8f2c2d6f0ce48a7b7217802db6135fe
I20260926 07:40:14.214250 19496 ts_tablet_manager.cc:1918] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 07:40:14.214361 19497 ts_tablet_manager.cc:1939] T d8f2c2d6f0ce48a7b7217802db6135fe P 50d613922f644c4baf64a1efbf22a099: Deleting consensus metadata
I20260926 07:40:14.214361 19498 ts_tablet_manager.cc:1939] T d8f2c2d6f0ce48a7b7217802db6135fe P b8f053c0d3784121bd705962089634b6: Deleting consensus metadata
I20260926 07:40:14.214426 19496 log.cc:1196] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/d8f2c2d6f0ce48a7b7217802db6135fe
I20260926 07:40:14.214859 19496 ts_tablet_manager.cc:1939] T d8f2c2d6f0ce48a7b7217802db6135fe P fe3b961f4ecf4831a0a4681d60a434a7: Deleting consensus metadata
I20260926 07:40:14.215394 18870 catalog_manager.cc:4841] TS 50d613922f644c4baf64a1efbf22a099 (127.11.9.66:40921): tablet d8f2c2d6f0ce48a7b7217802db6135fe (table test-table [id=0da845308d3648f99064f3dfa1fd871d]) successfully deleted
I20260926 07:40:14.215421 18871 catalog_manager.cc:4841] TS b8f053c0d3784121bd705962089634b6 (127.11.9.67:38687): tablet d8f2c2d6f0ce48a7b7217802db6135fe (table test-table [id=0da845308d3648f99064f3dfa1fd871d]) successfully deleted
I20260926 07:40:14.217334 18869 catalog_manager.cc:4841] TS fe3b961f4ecf4831a0a4681d60a434a7 (127.11.9.65:45277): tablet d8f2c2d6f0ce48a7b7217802db6135fe (table test-table [id=0da845308d3648f99064f3dfa1fd871d]) successfully deleted
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.65@KRBTEST.COM
I20260926 07:40:14.237313 19510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:14.207577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.11.9.65:45277)
0926 07:40:14.215526 (+ 7949us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:14.215534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:14.217976 (+ 2442us) server_negotiation.cc:407] Connection header received
0926 07:40:14.218193 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:14.218197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:14.218255 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:14.218361 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:14.219565 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.220344 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.221591 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.222003 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.222138 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:14.231706 (+ 9568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:14.231721 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:14.231739 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:14.231768 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:14.234628 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.235276 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.235283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.235285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.235400 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.235780 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.235789 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.235791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.236128 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:14.236429 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:14.236714 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 07:40:14.236919 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7795,"thread_start_us":7683,"threads_started":1}
I20260926 07:40:14.237990 19087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:14.333740 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18956
I20260926 07:40:14.352453 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19096
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.66@KRBTEST.COM
I20260926 07:40:14.366585 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:14.353234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.11.9.66:40921)
0926 07:40:14.353421 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:14.353429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:14.353453 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:40:14.353676 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:14.353689 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:14.353759 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:14.353865 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:14.355039 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.355751 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.356942 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.357476 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.357608 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:14.361381 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:14.361392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:14.361407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:14.361436 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:14.364014 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.364620 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.364626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.364629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.364749 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.365035 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.365038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.365040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.365651 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:14.365926 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:14.366152 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 07:40:14.366321 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:40:14.367288 19226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:14.489198 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19096
I20260926 07:40:14.511112 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19235
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[18840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.9.67@KRBTEST.COM
I20260926 07:40:14.525817 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:14.511875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.11.9.67:38687)
0926 07:40:14.512289 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:14.512293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:14.512309 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:40:14.512367 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:14.512370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:14.512425 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:14.512514 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:14.513826 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.514553 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.515728 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:14.516088 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:14.516174 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:14.520376 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:14.520389 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:14.520405 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:14.520447 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:14.523134 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.523761 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.523775 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.523777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.523888 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:14.524247 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:14.524254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:14.524258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:14.524603 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:14.524906 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:14.525249 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 07:40:14.525465 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":219,"threads_started":1}
I20260926 07:40:14.526448 19365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:14.668583 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19235
I20260926 07:40:14.692494 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 18853
I20260926 07:40:14.693645 18925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:14.808683 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 18853
2026-09-26T07:40:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5906 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[19519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[19519](info): set up 2 sockets
Sep 26 07:40:14 dist-test-slave-x492 krb5kdc[19519](info): commencing operation
krb5kdc: starting...
W20260926 07:40:15.914263 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.004s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:16 dist-test-slave-x492 krb5kdc[19519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:40:16.125597 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39729
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:39729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:16.442961 19532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:16.443323 19532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:16.443490 19532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:16.451181 19532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:16.451327 19532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:16.451447 19532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:16.451542 19532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:16.451635 19532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:16.461280 19532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:39729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:16.462567 19532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:16.463848 19532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:16.476150 19537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:16.476202 19538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:16.476229 19540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:16.477492 19532 server_base.cc:1034] running on GCE node
I20260926 07:40:16.478298 19532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:16.479718 19532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:16.480989 19532 hybrid_clock.cc:648] HybridClock initialized: now 1790408416480951 us; error 49 us; skew 500 ppm
I20260926 07:40:16.481391 19532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T07:40:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1742 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:40:16 dist-test-slave-x492 krb5kdc[19546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:40:16 dist-test-slave-x492 krb5kdc[19546](info): set up 2 sockets
Sep 26 07:40:16 dist-test-slave-x492 krb5kdc[19546](info): commencing operation
krb5kdc: starting...
W20260926 07:40:17.654318 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:17 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:40:17.864854 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:44565
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:44565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:18.183180 19559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:18.183447 19559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:18.183600 19559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:18.190582 19559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:18.190680 19559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:18.190761 19559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:18.190855 19559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:18.190940 19559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:18.200492 19559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:44565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:44565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:18.201802 19559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:18.203176 19559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:18.214903 19564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:18.214911 19565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:18.214911 19567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:18.215545 19559 server_base.cc:1034] running on GCE node
I20260926 07:40:18.216311 19559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:18.217751 19559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:18.218981 19559 hybrid_clock.cc:648] HybridClock initialized: now 1790408418218946 us; error 46 us; skew 500 ppm
I20260926 07:40:18.219319 19559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:18 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408418, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:18.222365 19559 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:40:18.223716 19559 webserver.cc:460] Webserver started at http://127.11.9.126:33959/ using document root <none> and password file <none>
I20260926 07:40:18.224336 19559 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:18.224431 19559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:18.224673 19559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:18.227509 19559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "d36d21488f7f4853a34ae3b7c923e44e"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "1d9d4b0bfb391ab0ec30b1f26ac2511f"
server_key_iv: "447e74a2f3e721151c5322c259864253"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.228180 19559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "d36d21488f7f4853a34ae3b7c923e44e"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "1d9d4b0bfb391ab0ec30b1f26ac2511f"
server_key_iv: "447e74a2f3e721151c5322c259864253"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.233125 19559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 07:40:18.236344 19574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:18.237697 19559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:40:18.237826 19559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "d36d21488f7f4853a34ae3b7c923e44e"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "1d9d4b0bfb391ab0ec30b1f26ac2511f"
server_key_iv: "447e74a2f3e721151c5322c259864253"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.237993 19559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:18.271037 19559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:18.275112 19559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:18.275339 19559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:18.292979 19559 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:44565
I20260926 07:40:18.293012 19625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:44565 every 8 connection(s)
I20260926 07:40:18.294337 19559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:18.297921 19626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:18.304652 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19559
I20260926 07:40:18.304883 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:18.305687 11301 external_mini_cluster.cc:1428] Setting key 37b76121d113309ac61a9bd840e87b35
I20260926 07:40:18.308921 19626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Bootstrap starting.
I20260926 07:40:18.312325 19626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:18.313509 19626 log.cc:824] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:18.316107 19626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: No bootstrap required, opened a new log
Sep 26 07:40:18 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:18.324474 19626 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:18.324852 19626 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:18.324971 19626 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36d21488f7f4853a34ae3b7c923e44e, State: Initialized, Role: FOLLOWER
I20260926 07:40:18.325698 19626 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:18.325989 19626 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:18.326138 19626 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:18.326315 19626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:18.328756 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:18.307708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.11.9.126:44565)
0926 07:40:18.308473 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:18.308485 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:18.308525 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:40:18.310286 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:18.310312 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:18.310715 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:18.311024 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:18.312913 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.314303 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.315525 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.316179 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.316392 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:18.320775 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:18.320784 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:18.320801 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:18.320842 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:18.323986 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:18.324667 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:18.324673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:18.324680 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:18.324886 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:18.325455 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:18.325462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:18.325465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:18.326187 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:18.326551 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:18.326926 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 07:40:18.327367 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":362,"threads_started":1}
I20260926 07:40:18.329612 19626 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:18.330142 19626 leader_election.cc:302] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36d21488f7f4853a34ae3b7c923e44e; no voters:
I20260926 07:40:18.330776 19626 leader_election.cc:288] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:18.330922 19631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:18.332338 19631 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 LEADER]: Becoming Leader. State: Replica: d36d21488f7f4853a34ae3b7c923e44e, State: Running, Role: LEADER
I20260926 07:40:18.332795 19631 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:18.332870 19626 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:18.335772 19633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36d21488f7f4853a34ae3b7c923e44e. Latest consensus state: current_term: 1 leader_uuid: "d36d21488f7f4853a34ae3b7c923e44e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } } }
I20260926 07:40:18.336117 19633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:18.336457 19632 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d36d21488f7f4853a34ae3b7c923e44e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } } }
I20260926 07:40:18.336787 19632 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:18.337945 19639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:18.345943 19639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:18.357044 19639 catalog_manager.cc:1347] Generated new cluster ID: 47254e0561814d7a85f7798cc2ded941
I20260926 07:40:18.357182 19639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:18.388723 19639 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:18.390137 19639 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:18.396052 19639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Generated new TSK 0
I20260926 07:40:18.397054 19639 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:40:18.449514 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:18.774420 19652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:18.774772 19652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:18.774959 19652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:18.782353 19652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:18.782514 19652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:18.782691 19652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:18.792287 19652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:18.793596 19652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:18.794910 19652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:18.808009 19657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:18.808156 19658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:18.808159 19660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:18.809448 19652 server_base.cc:1034] running on GCE node
I20260926 07:40:18.810102 19652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:18.811040 19652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:18.812238 19652 hybrid_clock.cc:648] HybridClock initialized: now 1790408418812194 us; error 36 us; skew 500 ppm
I20260926 07:40:18.812554 19652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:18 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408418, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:18.815634 19652 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:40:18.817077 19652 webserver.cc:460] Webserver started at http://127.11.9.65:37635/ using document root <none> and password file <none>
I20260926 07:40:18.817757 19652 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:18.817893 19652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:18.818221 19652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:18.820972 19652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "49eeb8087ee24079a35e7b49c6f68613"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "625d120ff6197eb85f105c94c00da5ea"
server_key_iv: "a28cc2ed25eadd7f4a03c3c2fed2f5ab"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.821816 19652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "49eeb8087ee24079a35e7b49c6f68613"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "625d120ff6197eb85f105c94c00da5ea"
server_key_iv: "a28cc2ed25eadd7f4a03c3c2fed2f5ab"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.826740 19652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:40:18.830125 19667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:18.831435 19652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:18.831602 19652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "49eeb8087ee24079a35e7b49c6f68613"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "625d120ff6197eb85f105c94c00da5ea"
server_key_iv: "a28cc2ed25eadd7f4a03c3c2fed2f5ab"
server_key_version: "encryptionkey@0"
I20260926 07:40:18.831807 19652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:18.846132 19652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:18.850199 19652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:18.850471 19652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:18.851398 19652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:18.852641 19652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:18.852751 19652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:18.852910 19652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:18.852986 19652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:18.883852 19652 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44181
I20260926 07:40:18.883896 19779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44181 every 8 connection(s)
I20260926 07:40:18.885136 19652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:18.891983 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19652
I20260926 07:40:18.892180 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:18.892803 11301 external_mini_cluster.cc:1428] Setting key 48773825dc335492753a76beea278fc0
Sep 26 07:40:18 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408418, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:18.905478 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:18.887718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:39383 (local address 127.11.9.126:44565)
0926 07:40:18.888144 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:18.888149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:18.888986 (+ 837us) server_negotiation.cc:407] Connection header received
0926 07:40:18.890816 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:18.890822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:18.890914 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:18.891042 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:18.892593 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.893440 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.895333 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.895870 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.896002 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:18.899614 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:18.899619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:18.899623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:18.899651 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:18.902147 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:18.903043 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:18.903047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:18.903049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:18.903162 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:18.903749 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:18.903753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:18.903755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:18.904037 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:18.904178 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:18.904982 (+ 804us) server_negotiation.cc:299] Negotiation successful
0926 07:40:18.905134 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":251,"threads_started":1}
I20260926 07:40:18.906896 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:18.888143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:39383)
0926 07:40:18.888821 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:18.888862 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:18.890550 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:18.891274 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:18.891282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:18.891769 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:18.892423 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:18.892442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.893599 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.893603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:18.895148 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:18.895163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:18.896086 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:18.896092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:18.896333 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:18.897033 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:18.897054 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:18.899394 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:18.902288 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:18.902300 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:18.902319 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:18.902910 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:18.903273 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:18.903281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:18.903288 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:18.903523 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:18.904171 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:18.904182 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:18.904624 (+ 442us) client_negotiation.cc:769] Sending connection context
0926 07:40:18.904928 (+ 304us) client_negotiation.cc:240] Negotiation successful
0926 07:40:18.905493 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":278,"threads_started":1}
I20260926 07:40:18.908495 19780 heartbeater.cc:347] Connected to a master server at 127.11.9.126:44565
I20260926 07:40:18.908811 19780 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:18.909689 19780 heartbeater.cc:511] Master 127.11.9.126:44565 requested a full tablet report, sending...
I20260926 07:40:18.912109 19591 ts_manager.cc:194] Registered new tserver with Master: 49eeb8087ee24079a35e7b49c6f68613 (127.11.9.65:44181)
I20260926 07:40:18.914368 19591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:39383
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:40:18.944270 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:19.264864 19786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:19.265125 19786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:19.265314 19786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:19.272222 19786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:19.272356 19786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:19.272550 19786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:19.282015 19786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:19.283217 19786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:19.284489 19786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:19.297120 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:19.297497 19794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:19.298256 19786 server_base.cc:1034] running on GCE node
W20260926 07:40:19.301452 19791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:19.302098 19786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:19.303090 19786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:19.304301 19786 hybrid_clock.cc:648] HybridClock initialized: now 1790408419304253 us; error 44 us; skew 500 ppm
I20260926 07:40:19.304615 19786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:19 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408419, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:19.310937 19786 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:40:19.312405 19786 webserver.cc:460] Webserver started at http://127.11.9.66:45697/ using document root <none> and password file <none>
I20260926 07:40:19.313069 19786 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:19.313235 19786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:19.313544 19786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:19.316273 19786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "d832ff01686e4b09834b0957ef5dc4c5"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "cf2a02125e1a6b3882caf1d28bb9581d"
server_key_iv: "acec01e14a46e01bf71c80c930b240d4"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.317027 19786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "d832ff01686e4b09834b0957ef5dc4c5"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "cf2a02125e1a6b3882caf1d28bb9581d"
server_key_iv: "acec01e14a46e01bf71c80c930b240d4"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.322247 19786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 07:40:19.325546 19801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.326774 19786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:19.326954 19786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "d832ff01686e4b09834b0957ef5dc4c5"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "cf2a02125e1a6b3882caf1d28bb9581d"
server_key_iv: "acec01e14a46e01bf71c80c930b240d4"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.327149 19786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:19.343573 19786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:19.347265 19786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:19.347566 19786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:19.348491 19786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:19.349772 19786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:19.349874 19786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.350028 19786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:19.350100 19786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.382062 19786 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:41695
I20260926 07:40:19.382143 19913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:41695 every 8 connection(s)
I20260926 07:40:19.383304 19786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:19.393577 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19786
I20260926 07:40:19.393769 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:19.394234 11301 external_mini_cluster.cc:1428] Setting key e500283874304112a8e0dbf8a1937237
Sep 26 07:40:19 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408419, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:19.403172 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:19.385866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:55109 (local address 127.11.9.126:44565)
0926 07:40:19.386034 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:19.386038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:19.387261 (+ 1223us) server_negotiation.cc:407] Connection header received
0926 07:40:19.388997 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:19.389002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:19.389092 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:19.389228 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:19.390623 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.391337 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.392821 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.393301 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.393433 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:19.397300 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:19.397305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:19.397308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:19.397337 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:19.399836 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:19.400727 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:19.400731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:19.400733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:19.400853 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:19.401335 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:19.401339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:19.401341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:19.401583 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:19.401702 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:19.402561 (+ 859us) server_negotiation.cc:299] Negotiation successful
0926 07:40:19.402802 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:40:19.403862 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:19.386369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:55109)
0926 07:40:19.387110 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:19.387154 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:19.388742 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:19.389455 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:19.389462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:19.389935 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:19.390460 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:19.390475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.391494 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.391499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:19.392683 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:19.392694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.393510 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.393516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:19.393811 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:19.394507 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:19.394527 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:19.397044 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:19.399969 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:19.399980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:19.399999 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:19.400618 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:19.400957 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:19.400964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:19.400970 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:19.401226 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:19.401707 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:19.401717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:19.402160 (+ 443us) client_negotiation.cc:769] Sending connection context
0926 07:40:19.402444 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 07:40:19.402807 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":333,"threads_started":1}
I20260926 07:40:19.405241 19914 heartbeater.cc:347] Connected to a master server at 127.11.9.126:44565
I20260926 07:40:19.405532 19914 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:19.406209 19914 heartbeater.cc:511] Master 127.11.9.126:44565 requested a full tablet report, sending...
I20260926 07:40:19.407646 19591 ts_manager.cc:194] Registered new tserver with Master: d832ff01686e4b09834b0957ef5dc4c5 (127.11.9.66:41695)
I20260926 07:40:19.408453 19591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:55109
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:40:19.442683 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:19.755007 19919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:19.755317 19919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:19.755483 19919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:19.762470 19919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:19.762615 19919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:19.762795 19919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:19.772056 19919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:19.773386 19919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:19.774668 19919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:19.787742 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:19.787813 19925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:19.787873 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:19.788538 19919 server_base.cc:1034] running on GCE node
I20260926 07:40:19.789084 19919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:19.790021 19919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:19.791235 19919 hybrid_clock.cc:648] HybridClock initialized: now 1790408419791198 us; error 54 us; skew 500 ppm
I20260926 07:40:19.791529 19919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:19 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408419, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:19.794531 19919 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:40:19.795939 19919 webserver.cc:460] Webserver started at http://127.11.9.67:36309/ using document root <none> and password file <none>
I20260926 07:40:19.796648 19919 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:19.796780 19919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:19.797073 19919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:19.799844 19919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "477579b5a6d1489a8bd522eff999fcc0"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "50ca22f0ba2e9a8d073f7bfe6d269c47"
server_key_iv: "569fb09cc00fe97db64ffffb85c71215"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.800623 19919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "477579b5a6d1489a8bd522eff999fcc0"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "50ca22f0ba2e9a8d073f7bfe6d269c47"
server_key_iv: "569fb09cc00fe97db64ffffb85c71215"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.805492 19919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260926 07:40:19.808813 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.810096 19919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:40:19.810261 19919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "477579b5a6d1489a8bd522eff999fcc0"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "50ca22f0ba2e9a8d073f7bfe6d269c47"
server_key_iv: "569fb09cc00fe97db64ffffb85c71215"
server_key_version: "encryptionkey@0"
I20260926 07:40:19.810452 19919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:19.822945 19919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:19.826969 19919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:19.827256 19919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:19.828146 19919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:19.829491 19919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:19.829576 19919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.829748 19919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:19.829800 19919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:19.861069 19919 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:46589
I20260926 07:40:19.861145 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:46589 every 8 connection(s)
I20260926 07:40:19.862366 19919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:19.871764 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 19919
I20260926 07:40:19.871943 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:19.872501 11301 external_mini_cluster.cc:1428] Setting key 7ae008da9004b0a72d1551d4470cb66d
Sep 26 07:40:19 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408419, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:19.880999 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:19.864905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:34235 (local address 127.11.9.126:44565)
0926 07:40:19.865085 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:19.865089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:19.866289 (+ 1200us) server_negotiation.cc:407] Connection header received
0926 07:40:19.867982 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:19.867987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:19.868092 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:19.868181 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:19.869592 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.870319 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.871791 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.872414 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.872552 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:19.876204 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:19.876209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:19.876213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:19.876256 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:19.878582 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:19.879220 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:19.879223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:19.879226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:19.879339 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:19.879691 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:19.879694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:19.879696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:19.879936 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:19.880044 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:19.880630 (+ 586us) server_negotiation.cc:299] Negotiation successful
0926 07:40:19.880774 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:40:19.882050 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:19.865458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:34235)
0926 07:40:19.866130 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:19.866168 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:19.867715 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:19.868368 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:19.868375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:19.868853 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:19.869450 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:19.869466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.870441 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.870444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:19.871614 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:19.871644 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:19.872653 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:19.872667 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:19.872848 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:19.873540 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:19.873561 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:19.876016 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:19.878713 (+ 2697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:19.878719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:19.878732 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:19.879115 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:19.879427 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:19.879430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:19.879432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:19.879597 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:19.880040 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:19.880046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:19.880319 (+ 273us) client_negotiation.cc:769] Sending connection context
0926 07:40:19.880536 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 07:40:19.880895 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":261,"threads_started":1}
I20260926 07:40:19.883414 20047 heartbeater.cc:347] Connected to a master server at 127.11.9.126:44565
I20260926 07:40:19.883713 20047 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:19.884428 20047 heartbeater.cc:511] Master 127.11.9.126:44565 requested a full tablet report, sending...
I20260926 07:40:19.885883 19591 ts_manager.cc:194] Registered new tserver with Master: 477579b5a6d1489a8bd522eff999fcc0 (127.11.9.67:46589)
I20260926 07:40:19.887013 19591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:34235
I20260926 07:40:19.887848 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.65@KRBTEST.COM" created.
I20260926 07:40:19.917832 19780 heartbeater.cc:503] Master 127.11.9.126:44565 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
Entry for principal oryx/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab.
WARNING: no policy specified for oryx/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
Entry for principal oryx/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab.
WARNING: no policy specified for oryx/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
Entry for principal oryx/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab.
I20260926 07:40:20.023101 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19652
Sep 26 07:40:20 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:40:20.040089 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:20.024302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33488 (local address 127.11.9.65:44181)
0926 07:40:20.025031 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:20.025041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:20.025076 (+ 35us) server_negotiation.cc:407] Connection header received
0926 07:40:20.025158 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:20.025189 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:20.025488 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:20.025729 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:20.027241 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.028454 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.029741 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.030334 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.030469 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:20.034586 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:20.034596 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:20.034614 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:20.034644 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:20.037359 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.038036 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.038042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.038044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.038164 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.038517 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.038523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.038527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.038857 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:20.039160 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:20.039463 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 07:40:20.039735 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":897,"threads_started":1}
I20260926 07:40:20.041447 19775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:20.123627 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19652
I20260926 07:40:20.140115 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19786
Sep 26 07:40:20 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:40:20.155761 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:20.140978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.11.9.66:41695)
0926 07:40:20.141548 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:20.141555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:20.141581 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:40:20.141641 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:20.141647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:20.141855 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:20.141982 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:20.143244 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.144402 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.145725 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.146340 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.146444 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:20.150671 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:20.150681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:20.150694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:20.150724 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:20.153416 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.153990 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.153996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.153999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.154135 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.154426 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.154428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.154430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.154726 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:20.154935 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:20.155201 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 07:40:20.155430 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":310,"threads_started":1}
I20260926 07:40:20.157055 19909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:20.238592 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19786
I20260926 07:40:20.254743 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19919
Sep 26 07:40:20 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:40:20.271045 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:20.255771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.11.9.67:46589)
0926 07:40:20.256393 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:20.256400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:20.256427 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:40:20.256483 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:20.256489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:20.256801 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:20.256970 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:20.258306 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.259307 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.260531 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:20.261066 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:20.261241 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:20.265702 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:20.265713 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:20.265726 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:20.265755 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:20.268283 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.268936 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.268942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.268944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.269079 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:20.269544 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:20.269552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:20.269557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:20.269872 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:20.270199 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:20.270479 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 07:40:20.270727 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":354,"threads_started":1}
I20260926 07:40:20.272421 20042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:20.354801 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19919
I20260926 07:40:20.370568 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 19559
I20260926 07:40:20.371838 19621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:20.473701 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 19559
I20260926 07:40:20.494294 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:44565
--webserver_interface=127.11.9.126
--webserver_port=33959
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:44565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:20.815749 20064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:20.816026 20064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:20.816114 20064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:20.823161 20064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:20.823262 20064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:20.823307 20064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:20.823380 20064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:20.823436 20064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:20.832998 20064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:44565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:44565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=33959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:20.834352 20064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:20.835570 20064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:20.847186 20069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:20.847127 20072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:20.847098 20070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:20.847885 20064 server_base.cc:1034] running on GCE node
I20260926 07:40:20.848565 20064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:20.849857 20064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:20.851063 20064 hybrid_clock.cc:648] HybridClock initialized: now 1790408420850998 us; error 53 us; skew 500 ppm
I20260926 07:40:20.851395 20064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:20 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408420, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:20.854408 20064 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:40:20.855856 20064 webserver.cc:460] Webserver started at http://127.11.9.126:33959/ using document root <none> and password file <none>
I20260926 07:40:20.856556 20064 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:20.856704 20064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:20.862030 20064 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:20.864796 20079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:20.866160 20064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:20.866330 20064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "d36d21488f7f4853a34ae3b7c923e44e"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "1d9d4b0bfb391ab0ec30b1f26ac2511f"
server_key_iv: "447e74a2f3e721151c5322c259864253"
server_key_version: "encryptionkey@0"
I20260926 07:40:20.867226 20064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:20.883719 20064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:20.887416 20064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:20.887679 20064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:20.905696 20064 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:44565
I20260926 07:40:20.905743 20130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:44565 every 8 connection(s)
I20260926 07:40:20.907102 20064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:20.913867 20131 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:40:20.915627 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20064
I20260926 07:40:20.916405 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:44181
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:44565
--webserver_port=37635
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:40:20.917318 20131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Bootstrap starting.
I20260926 07:40:20.942113 20131 log.cc:824] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:20.952858 20131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:40:20.953467 20131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Bootstrap complete.
I20260926 07:40:20.966343 20131 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:20.967437 20131 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36d21488f7f4853a34ae3b7c923e44e, State: Initialized, Role: FOLLOWER
I20260926 07:40:20.968135 20131 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:20.968425 20131 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:20.968560 20131 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:20.968778 20131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:40:20.974705 20131 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:20.975270 20131 leader_election.cc:302] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36d21488f7f4853a34ae3b7c923e44e; no voters:
I20260926 07:40:20.975929 20131 leader_election.cc:288] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 07:40:20.976081 20135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:40:20.977609 20135 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [term 2 LEADER]: Becoming Leader. State: Replica: d36d21488f7f4853a34ae3b7c923e44e, State: Running, Role: LEADER
I20260926 07:40:20.978039 20131 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:20.978188 20135 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } }
I20260926 07:40:20.980386 20137 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36d21488f7f4853a34ae3b7c923e44e. Latest consensus state: current_term: 2 leader_uuid: "d36d21488f7f4853a34ae3b7c923e44e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } } }
I20260926 07:40:20.980778 20137 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:20.981539 20136 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d36d21488f7f4853a34ae3b7c923e44e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36d21488f7f4853a34ae3b7c923e44e" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 44565 } } }
I20260926 07:40:20.981900 20136 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:20.989324 20140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:20.995048 20140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:20.996353 20140 catalog_manager.cc:1259] Loaded cluster ID: 47254e0561814d7a85f7798cc2ded941
I20260926 07:40:20.996443 20140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:21.000396 20140 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:21.001786 20140 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d36d21488f7f4853a34ae3b7c923e44e: Loaded TSK: 0
I20260926 07:40:21.003208 20140 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 07:40:21.266304 20133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:21.266620 20133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:21.266777 20133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:21.274178 20133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:21.274327 20133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:21.274514 20133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:21.284472 20133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.65.keytab
--principal=oryx/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:44181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=37635
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:21.285924 20133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:21.287249 20133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:21.300791 20158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:21.300872 20157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:21.301584 20160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:21.302443 20133 server_base.cc:1034] running on GCE node
I20260926 07:40:21.303010 20133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:21.304026 20133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:21.305266 20133 hybrid_clock.cc:648] HybridClock initialized: now 1790408421305196 us; error 55 us; skew 500 ppm
I20260926 07:40:21.305591 20133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408421, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:21.308941 20133 init.cc:377] Logged in from keytab as oryx/127.11.9.65@KRBTEST.COM (short username oryx)
I20260926 07:40:21.310640 20133 webserver.cc:460] Webserver started at http://127.11.9.65:37635/ using document root <none> and password file <none>
I20260926 07:40:21.311337 20133 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:21.311492 20133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:21.317090 20133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:21.319923 20167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.321368 20133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:40:21.321534 20133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "49eeb8087ee24079a35e7b49c6f68613"
format_stamp: "Formatted at 2026-09-26 07:40:18 on dist-test-slave-x492"
server_key: "625d120ff6197eb85f105c94c00da5ea"
server_key_iv: "a28cc2ed25eadd7f4a03c3c2fed2f5ab"
server_key_version: "encryptionkey@0"
I20260926 07:40:21.322500 20133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:21.341403 20133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:21.345212 20133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:21.345484 20133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:21.346458 20133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:21.347740 20133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:21.347862 20133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.347998 20133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:21.348089 20133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.379801 20133 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44181
I20260926 07:40:21.379843 20279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44181 every 8 connection(s)
I20260926 07:40:21.381379 20133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:21.391218 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20133
I20260926 07:40:21.391816 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:41695
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:44565
--webserver_port=45697
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:21.400138 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.384483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:43783)
0926 07:40:21.385339 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.385381 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.386926 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.388248 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.388255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.388746 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.389367 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.389386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.391106 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.391117 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.392536 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.392549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.393639 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.393647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.393896 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.394571 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.394603 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.399250 (+ 4647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":372,"threads_started":1}
W20260926 07:40:21.400833 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.384441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:43783 (local address 127.11.9.126:44565)
0926 07:40:21.385203 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.385219 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.385510 (+ 291us) server_negotiation.cc:407] Connection header received
0926 07:40:21.387465 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.387486 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.387800 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.388080 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.389537 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.390951 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.392676 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.393390 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.393596 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.399677 (+ 6081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:43783: BlockingRecv error: recv got EOF from 127.11.9.65:43783 (error 108)
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":260,"threads_started":1}
W20260926 07:40:21.400889 20280 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:44565 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.9.126:44565: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:21.408949 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.401529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:53211)
0926 07:40:21.401692 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.401710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.401824 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.402164 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.402169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.402505 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.402629 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.402637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.403636 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.403642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.404510 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.404521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.405213 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.405219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.405382 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.405968 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.405985 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.408679 (+ 2694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 07:40:21.409085 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.401726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:53211 (local address 127.11.9.126:44565)
0926 07:40:21.401908 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.401915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.401930 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:40:21.401974 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.401987 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.402038 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.402140 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.402766 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.403497 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.404647 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.405040 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.405171 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.408870 (+ 3699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:53211: BlockingRecv error: recv got EOF from 127.11.9.65:53211 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:21.417727 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.409666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:35633)
0926 07:40:21.409816 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.409840 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.409912 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.410273 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.410276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.410515 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.410644 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.410651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.412164 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.412167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.413044 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.413054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.414087 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.414091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.414232 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.414780 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.414796 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.417441 (+ 2645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 07:40:21.417883 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.409774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:35633 (local address 127.11.9.126:44565)
0926 07:40:21.409960 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.409968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.409988 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:40:21.410043 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.410050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.410139 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.410255 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.410782 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.412017 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.413211 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.413954 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.414104 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.417644 (+ 3540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:35633: BlockingRecv error: recv got EOF from 127.11.9.65:35633 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 07:40:21.418150 20280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:40:21.711361 20284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:21.711617 20284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:21.711705 20284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:21.718712 20284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:21.718834 20284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:21.718943 20284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:21.728478 20284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.66.keytab
--principal=oryx/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:41695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=45697
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:21.729686 20284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:21.730880 20284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:21.743333 20292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:21.743425 20290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:21.743454 20289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:21.744359 20284 server_base.cc:1034] running on GCE node
I20260926 07:40:21.744949 20284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:21.745893 20284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:21.747109 20284 hybrid_clock.cc:648] HybridClock initialized: now 1790408421747055 us; error 47 us; skew 500 ppm
I20260926 07:40:21.747449 20284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408421, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:21.750613 20284 init.cc:377] Logged in from keytab as oryx/127.11.9.66@KRBTEST.COM (short username oryx)
I20260926 07:40:21.751999 20284 webserver.cc:460] Webserver started at http://127.11.9.66:45697/ using document root <none> and password file <none>
I20260926 07:40:21.752676 20284 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:21.752830 20284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:21.757893 20284 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:21.760421 20299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.761745 20284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:21.761910 20284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "d832ff01686e4b09834b0957ef5dc4c5"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "cf2a02125e1a6b3882caf1d28bb9581d"
server_key_iv: "acec01e14a46e01bf71c80c930b240d4"
server_key_version: "encryptionkey@0"
I20260926 07:40:21.762826 20284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:21.793862 20284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:21.797714 20284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:21.797991 20284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:21.798947 20284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:21.800235 20284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:21.800352 20284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.800487 20284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:21.800580 20284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:21.833112 20284 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:41695
I20260926 07:40:21.833226 20411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:41695 every 8 connection(s)
I20260926 07:40:21.834576 20284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:21.844537 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20284
I20260926 07:40:21.845127 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:46589
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:44565
--webserver_port=36309
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 07:40:21.850163 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.836900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:36623 (local address 127.11.9.126:44565)
0926 07:40:21.837068 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.837073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.838295 (+ 1222us) server_negotiation.cc:407] Connection header received
0926 07:40:21.840104 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.840110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.840201 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.840320 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.841801 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.842506 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.844219 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.844613 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.844744 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.849986 (+ 5242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:36623: BlockingRecv error: recv got EOF from 127.11.9.66:36623 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 07:40:21.851096 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.837404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:36623)
0926 07:40:21.838114 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.838156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.839845 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.840544 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.840552 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.841035 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.841632 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.841649 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.842677 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.842681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.844050 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.844062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.844845 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.844852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.845063 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.845801 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.845822 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.849847 (+ 4025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":293,"threads_started":1}
W20260926 07:40:21.851881 20412 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:44565 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.9.126:44565: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:21.861624 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.852385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:34747)
0926 07:40:21.852598 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.852618 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.852713 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.853002 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.853010 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.853463 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.853662 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.853676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.854653 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.854660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.856286 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.856303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.856920 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.856928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.857190 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.857818 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.857859 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.861285 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 07:40:21.861622 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.852528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:34747 (local address 127.11.9.126:44565)
0926 07:40:21.852670 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.852674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.852690 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:40:21.852817 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.852820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.852875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.852950 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.853799 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.854497 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.856416 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.856775 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.856867 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.861436 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:34747: BlockingRecv error: recv got EOF from 127.11.9.66:34747 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:21 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:21.870749 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:21.862391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:42751)
0926 07:40:21.862634 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:21.862659 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:21.862729 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:21.863052 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:21.863055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:21.863309 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:21.863436 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.863443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.864904 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.864907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.865935 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:21.865946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.867010 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.867014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:21.867202 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.867750 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:21.867766 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:21.870473 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260926 07:40:21.871012 20412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:40:21.871044 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:21.862544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:42751 (local address 127.11.9.126:44565)
0926 07:40:21.862753 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:21.862762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:21.862783 (+ 21us) server_negotiation.cc:407] Connection header received
0926 07:40:21.862838 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:21.862846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:21.862926 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:21.863045 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:21.863570 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.864778 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.866114 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:21.866880 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:21.867026 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:21.870709 (+ 3683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:42751: BlockingRecv error: recv got EOF from 127.11.9.66:42751 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 07:40:22.179754 20415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:22.180006 20415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:22.180171 20415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:22.187223 20415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:22.187319 20415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:22.187428 20415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:22.196805 20415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/krb5kdc/oryx_127.11.9.67.keytab
--principal=oryx/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:46589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=36309
--tserver_master_addrs=127.11.9.126:44565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:22.198064 20415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:22.199318 20415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:22.212031 20423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:22.211992 20420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:22.212551 20421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:22.212783 20415 server_base.cc:1034] running on GCE node
I20260926 07:40:22.213438 20415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:22.214278 20415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:22.215504 20415 hybrid_clock.cc:648] HybridClock initialized: now 1790408422215451 us; error 55 us; skew 500 ppm
I20260926 07:40:22.215814 20415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408422, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:22.218845 20415 init.cc:377] Logged in from keytab as oryx/127.11.9.67@KRBTEST.COM (short username oryx)
I20260926 07:40:22.220180 20415 webserver.cc:460] Webserver started at http://127.11.9.67:36309/ using document root <none> and password file <none>
I20260926 07:40:22.220870 20415 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:22.221020 20415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:22.226300 20415 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 07:40:22.228973 20430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:22.230424 20415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:22.230605 20415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "477579b5a6d1489a8bd522eff999fcc0"
format_stamp: "Formatted at 2026-09-26 07:40:19 on dist-test-slave-x492"
server_key: "50ca22f0ba2e9a8d073f7bfe6d269c47"
server_key_iv: "569fb09cc00fe97db64ffffb85c71215"
server_key_version: "encryptionkey@0"
I20260926 07:40:22.231495 20415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:22.261965 20415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:22.266047 20415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:22.266314 20415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:22.267266 20415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:22.268510 20415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:22.268632 20415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:22.268754 20415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:22.268841 20415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:22.300868 20415 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:46589
I20260926 07:40:22.300954 20542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:46589 every 8 connection(s)
I20260926 07:40:22.302430 20415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:22.309535 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20415
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 07:40:22.319963 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:22.305041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:53009 (local address 127.11.9.126:44565)
0926 07:40:22.305267 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.305273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.306319 (+ 1046us) server_negotiation.cc:407] Connection header received
0926 07:40:22.308138 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.308143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.308209 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.308295 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.310048 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.311262 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.313665 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.314338 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.314458 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.319672 (+ 5214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:53009: BlockingRecv error: recv got EOF from 127.11.9.67:53009 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:40:22.320674 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.305432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:53009)
0926 07:40:22.306153 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:22.306191 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:22.307897 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:22.308501 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:22.308509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:22.308995 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:22.309842 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.309869 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.311434 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.311440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.313534 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.313546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.314525 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.314529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.314769 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.315456 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:22.315483 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:22.319476 (+ 3993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":271,"threads_started":1}
W20260926 07:40:22.321391 20543 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:44565 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.9.126:44565: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
I20260926 07:40:22.326896 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.310594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.11.9.126:44565)
0926 07:40:22.311314 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.311321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.311358 (+ 37us) server_negotiation.cc:407] Connection header received
0926 07:40:22.311404 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.311414 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.311465 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.311546 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.313457 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.314608 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.316360 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.316770 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.316903 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.319478 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:22.319487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:22.319502 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:22.319535 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:22.323162 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:22.323949 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:22.323955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:22.323969 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:22.324266 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:22.324693 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:22.324698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:22.324701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:22.325426 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:22.325772 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:22.326184 (+ 412us) server_negotiation.cc:299] Negotiation successful
0926 07:40:22.326536 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":495,"threads_started":1}
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:22.330632 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.321929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:45209)
0926 07:40:22.322127 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:22.322145 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:22.322246 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:22.322531 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:22.322535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:22.322928 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:22.323103 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.323114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.324506 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.324510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.325753 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.325764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.326717 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.326723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.326883 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.327401 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:22.327416 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:22.330372 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 07:40:22.330742 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:22.322050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:45209 (local address 127.11.9.126:44565)
0926 07:40:22.322191 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.322195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.322209 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:40:22.322351 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.322355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.322402 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.322476 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.323256 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.324319 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.325868 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.326540 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.326705 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.330512 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:45209: BlockingRecv error: recv got EOF from 127.11.9.67:45209 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:22.338231 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.331321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:59815)
0926 07:40:22.331472 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:22.331489 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:22.331567 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:22.331879 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:22.331882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:22.332163 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:22.332293 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.332300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.333267 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.333272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.334154 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.334164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.334762 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.334766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.334886 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.335342 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:22.335357 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:22.338043 (+ 2686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 07:40:22.338436 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:22.331495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:59815 (local address 127.11.9.126:44565)
0926 07:40:22.331673 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.331677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.331691 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:40:22.331727 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.331730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.331775 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.331847 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.332405 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.333098 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.334283 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.334654 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.334743 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.338200 (+ 3457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:59815: BlockingRecv error: recv got EOF from 127.11.9.67:59815 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260926 07:40:22.338587 20543 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:22.427495 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.418688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:49677)
0926 07:40:22.419256 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:22.419273 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:22.419432 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:22.419767 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:22.419772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:22.420258 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:22.420442 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.420450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.421505 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.421510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.422465 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.422476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.423227 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.423234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.423402 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.424000 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:22.424045 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:22.427151 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":297,"threads_started":1}
W20260926 07:40:22.427567 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:22.418838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:49677 (local address 127.11.9.126:44565)
0926 07:40:22.419074 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.419081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.419363 (+ 282us) server_negotiation.cc:407] Connection header received
0926 07:40:22.419574 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.419578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.419636 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.419745 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.420589 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.421360 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.422609 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.423022 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.423155 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.427354 (+ 4199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:49677: BlockingRecv error: recv got EOF from 127.11.9.65:49677 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:22 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:22.880012 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:22.871608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:45325)
0926 07:40:22.872110 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:22.872127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:22.872235 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:22.872584 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:22.872589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:22.873273 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:22.873471 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.873481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.874442 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.874449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.875394 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:22.875405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.876172 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.876179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:22.876366 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.876963 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:22.876980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:22.879799 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":280,"threads_started":1}
W20260926 07:40:22.880236 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:22.871635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:45325 (local address 127.11.9.126:44565)
0926 07:40:22.871805 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:22.871810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:22.872212 (+ 402us) server_negotiation.cc:407] Connection header received
0926 07:40:22.872376 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:22.872380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:22.872441 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:22.872561 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:22.873615 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.874307 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.875545 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:22.875991 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:22.876115 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:22.879979 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:45325: BlockingRecv error: recv got EOF from 127.11.9.66:45325 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:23.347713 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:23.339115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:46269)
0926 07:40:23.339634 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:23.339650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:23.339774 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:23.340107 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:23.340110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:23.340536 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:23.340784 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.340797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.341864 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.341868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.342923 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.342936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.343827 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.343836 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.344025 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.344600 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:23.344617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:23.347367 (+ 2750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":292,"threads_started":1}
W20260926 07:40:23.347793 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:23.339267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:46269 (local address 127.11.9.126:44565)
0926 07:40:23.339437 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:23.339442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:23.339727 (+ 285us) server_negotiation.cc:407] Connection header received
0926 07:40:23.339907 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:23.339911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:23.339972 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:23.340066 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:23.340932 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.341741 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.343064 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.343619 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.343761 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.347578 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:46269: BlockingRecv error: recv got EOF from 127.11.9.67:46269 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:23.436115 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:23.428410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:46013)
0926 07:40:23.428931 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:23.428947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:23.429054 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:23.429439 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:23.429443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:23.429805 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:23.429945 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.429953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.430903 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.430907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.431793 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.431804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.432414 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.432418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.432522 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.433126 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:23.433143 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:23.435744 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":287,"threads_started":1}
W20260926 07:40:23.436190 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:23.428454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:46013 (local address 127.11.9.126:44565)
0926 07:40:23.428621 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:23.428627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:23.429021 (+ 394us) server_negotiation.cc:407] Connection header received
0926 07:40:23.429147 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:23.429150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:23.429312 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:23.429394 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:23.430063 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.430790 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.431911 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.432288 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.432420 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.435926 (+ 3506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:46013: BlockingRecv error: recv got EOF from 127.11.9.65:46013 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:23 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:23.888921 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:23.880848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:40227)
0926 07:40:23.881473 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:23.881489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:23.881657 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:23.882003 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:23.882008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:23.882412 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:23.882550 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.882558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.883590 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.883597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.884477 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:23.884488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.885377 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.885383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:23.885519 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.886154 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:23.886172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:23.888692 (+ 2520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":406,"threads_started":1}
W20260926 07:40:23.889071 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:23.880925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40227 (local address 127.11.9.126:44565)
0926 07:40:23.881267 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:23.881276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:23.881579 (+ 303us) server_negotiation.cc:407] Connection header received
0926 07:40:23.881803 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:23.881807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:23.881868 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:23.881972 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:23.882699 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.883448 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.884619 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:23.885230 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:23.885363 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:23.888872 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:40227: BlockingRecv error: recv got EOF from 127.11.9.66:40227 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:24.360141 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:24.348744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:48711)
0926 07:40:24.349661 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:24.349701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:24.349843 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:24.350175 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:24.350179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:24.350553 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:24.350687 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.350695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.351640 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.351643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.352509 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.352519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.353140 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.353144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.353316 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.357148 (+ 3832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:24.357181 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:24.359811 (+ 2630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":676,"threads_started":1}
W20260926 07:40:24.360241 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:24.348944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:48711 (local address 127.11.9.126:44565)
0926 07:40:24.349203 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:24.349212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:24.349783 (+ 571us) server_negotiation.cc:407] Connection header received
0926 07:40:24.349975 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:24.349980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:24.350052 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:24.350126 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:24.350810 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.351525 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.352621 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.353023 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.353110 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.359992 (+ 6882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:48711: BlockingRecv error: recv got EOF from 127.11.9.67:48711 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:24.445091 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:24.437135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:49423)
0926 07:40:24.437863 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:24.437886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:24.438013 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:24.438320 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:24.438323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:24.438631 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:24.438788 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.438796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.439722 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.439725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.440592 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.440603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.441215 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.441221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.441360 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.441886 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:24.441902 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:24.444850 (+ 2948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":476,"threads_started":1}
W20260926 07:40:24.445291 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:24.437243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:49423 (local address 127.11.9.126:44565)
0926 07:40:24.437491 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:24.437499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:24.437976 (+ 477us) server_negotiation.cc:407] Connection header received
0926 07:40:24.438124 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:24.438127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:24.438200 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:24.438276 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:24.438910 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.439609 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.440708 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.441057 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.441142 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.445014 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:49423: BlockingRecv error: recv got EOF from 127.11.9.65:49423 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:24 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:24.897920 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:24.889824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:45755)
0926 07:40:24.890333 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:24.890347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:24.890536 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:24.890939 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:24.890945 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:24.891387 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:24.891635 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.891648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.892740 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.892744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.893652 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:24.893662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.894286 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.894290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:24.894405 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.895023 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:24.895049 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:24.897662 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":261,"threads_started":1}
W20260926 07:40:24.898121 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:24.889890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:45755 (local address 127.11.9.126:44565)
0926 07:40:24.890114 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:24.890123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:24.890441 (+ 318us) server_negotiation.cc:407] Connection header received
0926 07:40:24.890725 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:24.890732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:24.890814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:24.890890 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:24.891801 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.892627 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.893769 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:24.894164 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:24.894259 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:24.897875 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:45755: BlockingRecv error: recv got EOF from 127.11.9.66:45755 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:25.369496 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:25.361135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:54551)
0926 07:40:25.361670 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:25.361685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:25.361845 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:25.362229 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:25.362235 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:25.362705 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:25.362871 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.362880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.364029 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.364033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.364900 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.364911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.365537 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.365541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.365710 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.366259 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:25.366278 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:25.369091 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":263,"threads_started":1}
W20260926 07:40:25.369606 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:25.361316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:54551 (local address 127.11.9.126:44565)
0926 07:40:25.361527 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:25.361533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:25.361792 (+ 259us) server_negotiation.cc:407] Connection header received
0926 07:40:25.362001 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:25.362007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:25.362084 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:25.362188 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:25.363011 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.363908 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.365013 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.365430 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.365513 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.369405 (+ 3892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:54551: BlockingRecv error: recv got EOF from 127.11.9.67:54551 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:25.454007 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:25.446068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:52615)
0926 07:40:25.446573 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:25.446589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:25.446694 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:25.446996 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:25.446999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:25.447362 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:25.447504 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.447511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.448518 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.448522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.449541 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.449554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.450232 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.450236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.450354 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.450898 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:25.450915 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:25.453676 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":288,"threads_started":1}
W20260926 07:40:25.454128 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:25.446147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:52615 (local address 127.11.9.126:44565)
0926 07:40:25.446363 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:25.446370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:25.446665 (+ 295us) server_negotiation.cc:407] Connection header received
0926 07:40:25.446791 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:25.446795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:25.446856 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:25.446933 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:25.447653 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.448404 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.449669 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.450107 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.450207 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.453859 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:52615: BlockingRecv error: recv got EOF from 127.11.9.65:52615 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:25 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:25.906862 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:25.898805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:33323)
0926 07:40:25.899302 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:25.899318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:25.899425 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:25.899709 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:25.899712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:25.900053 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:25.900198 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.900207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.901108 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.901111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.902112 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:25.902125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.902742 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.902746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:25.902869 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.903479 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:25.903496 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:25.906548 (+ 3052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":275,"threads_started":1}
W20260926 07:40:25.906934 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:25.898843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:33323 (local address 127.11.9.126:44565)
0926 07:40:25.899010 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:25.899015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:25.899387 (+ 372us) server_negotiation.cc:407] Connection header received
0926 07:40:25.899516 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:25.899520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:25.899592 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:25.899672 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:25.900312 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.901002 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.902235 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:25.902626 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:25.902723 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:25.906696 (+ 3973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:33323: BlockingRecv error: recv got EOF from 127.11.9.66:33323 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:26.378703 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:26.370388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:43533)
0926 07:40:26.370821 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:26.370837 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:26.370942 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:26.371253 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:26.371257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:26.371660 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:26.371803 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.371811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.372826 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.372834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.373811 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.373822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.374534 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.374540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.374716 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.375371 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:26.375397 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:26.378471 (+ 3074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":232,"threads_started":1}
W20260926 07:40:26.378965 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:26.370475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:43533 (local address 127.11.9.126:44565)
0926 07:40:26.370701 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:26.370707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:26.370908 (+ 201us) server_negotiation.cc:407] Connection header received
0926 07:40:26.371033 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:26.371037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:26.371095 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:26.371207 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:26.371943 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.372692 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.373951 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.374381 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.374514 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.378688 (+ 4174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:43533: BlockingRecv error: recv got EOF from 127.11.9.67:43533 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:26.463447 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:26.454961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:58553)
0926 07:40:26.455433 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:26.455449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:26.455554 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:26.455877 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:26.455880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:26.456335 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:26.456513 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.456526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.458009 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.458015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.459068 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.459081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.459789 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.459794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.459961 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.460582 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:26.460599 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:26.463218 (+ 2619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":255,"threads_started":1}
W20260926 07:40:26.463610 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:26.455060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:58553 (local address 127.11.9.126:44565)
0926 07:40:26.455299 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:26.455305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:26.455531 (+ 226us) server_negotiation.cc:407] Connection header received
0926 07:40:26.455648 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:26.455651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:26.455745 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:26.455822 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:26.456750 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.457852 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.459209 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.459633 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.459780 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.463391 (+ 3611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:58553: BlockingRecv error: recv got EOF from 127.11.9.65:58553 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:26 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:26.916067 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:26.907868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:55949)
0926 07:40:26.908423 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:26.908441 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:26.908601 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:26.908941 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:26.908945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:26.909426 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:26.909602 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.909611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.910633 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.910640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.911537 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:26.911548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.912249 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.912254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:26.912429 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.913038 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:26.913055 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:26.915840 (+ 2785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":350,"threads_started":1}
W20260926 07:40:26.916255 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:26.908010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:55949 (local address 127.11.9.126:44565)
0926 07:40:26.908208 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:26.908215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:26.908537 (+ 322us) server_negotiation.cc:407] Connection header received
0926 07:40:26.908739 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:26.908745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:26.908814 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:26.908910 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:26.909750 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.910495 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.911683 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:26.912103 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:26.912227 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:26.916013 (+ 3786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:55949: BlockingRecv error: recv got EOF from 127.11.9.66:55949 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:27.388463 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:27.379630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:47227)
0926 07:40:27.380155 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:27.380178 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:27.380376 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:27.380878 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:27.380883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:27.381433 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:27.381591 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.381599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.382931 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.382934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.383900 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.383910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.384535 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.384539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.384648 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.385276 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:27.385293 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:27.388129 (+ 2836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":308,"threads_started":1}
W20260926 07:40:27.388566 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:27.379712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:47227 (local address 127.11.9.126:44565)
0926 07:40:27.379931 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:27.379940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:27.380271 (+ 331us) server_negotiation.cc:407] Connection header received
0926 07:40:27.380516 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:27.380522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:27.380597 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:27.380690 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:27.381732 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.382790 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.384022 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.384420 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.384512 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.388297 (+ 3785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:47227: BlockingRecv error: recv got EOF from 127.11.9.67:47227 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:27.472250 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:27.464323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:38021)
0926 07:40:27.464859 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:27.464875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:27.464993 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:27.465312 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:27.465315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:27.465734 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:27.465898 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.465907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.466977 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.466981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.467859 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.467869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.468491 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.468495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.468605 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.469276 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:27.469293 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:27.472030 (+ 2737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":300,"threads_started":1}
W20260926 07:40:27.472450 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:27.464363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38021 (local address 127.11.9.126:44565)
0926 07:40:27.464527 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:27.464531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:27.464957 (+ 426us) server_negotiation.cc:407] Connection header received
0926 07:40:27.465101 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:27.465104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:27.465163 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:27.465262 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:27.466016 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.466850 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.467970 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.468373 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.468456 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.472221 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:38021: BlockingRecv error: recv got EOF from 127.11.9.65:38021 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:27 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:27.924901 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:27.917008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:34397)
0926 07:40:27.917496 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:27.917513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:27.917620 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:27.917913 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:27.917917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:27.918290 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:27.918436 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.918444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.919396 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.919399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.920534 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:27.920545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.921352 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.921364 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:27.921566 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.922213 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:27.922230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:27.924696 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":311,"threads_started":1}
W20260926 07:40:27.925115 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:27.917165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:34397 (local address 127.11.9.126:44565)
0926 07:40:27.917390 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:27.917398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:27.917590 (+ 192us) server_negotiation.cc:407] Connection header received
0926 07:40:27.917712 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:27.917716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:27.917776 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:27.917875 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:27.918566 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.919289 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.920677 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:27.921203 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:27.921292 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:27.924899 (+ 3607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:34397: BlockingRecv error: recv got EOF from 127.11.9.66:34397 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:28.397670 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:28.389579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:43723)
0926 07:40:28.390107 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:28.390122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:28.390271 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:28.390670 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:28.390675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:28.391239 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:28.391469 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.391482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.392565 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.392568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.393471 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.393482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.394142 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.394146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.394257 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.394879 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:28.394895 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:28.397465 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":267,"threads_started":1}
W20260926 07:40:28.397930 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:28.389627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:43723 (local address 127.11.9.126:44565)
0926 07:40:28.389819 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:28.389827 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:28.390200 (+ 373us) server_negotiation.cc:407] Connection header received
0926 07:40:28.390412 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:28.390418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:28.390513 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:28.390615 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:28.391627 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.392443 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.393592 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.394019 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.394111 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.397634 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:43723: BlockingRecv error: recv got EOF from 127.11.9.67:43723 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:28.481441 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:28.473156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:35505)
0926 07:40:28.473875 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:28.473891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:28.474000 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:28.474287 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:28.474290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:28.474659 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:28.474827 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.474835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.475740 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.475744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.476625 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.476635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.477320 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.477324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.477442 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.478034 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:28.478051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:28.481094 (+ 3043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":345,"threads_started":1}
W20260926 07:40:28.481623 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:28.473288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:35505 (local address 127.11.9.126:44565)
0926 07:40:28.473469 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:28.473477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:28.473974 (+ 497us) server_negotiation.cc:407] Connection header received
0926 07:40:28.474099 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:28.474103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:28.474166 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:28.474242 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:28.474946 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.475628 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.476740 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.477142 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.477299 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.481463 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:35505: BlockingRecv error: recv got EOF from 127.11.9.65:35505 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:28 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:28.933880 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:28.925798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:51965)
0926 07:40:28.926378 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:28.926394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:28.926499 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:28.926804 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:28.926807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:28.927193 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:28.927363 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.927372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.928330 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.928334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.929255 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:28.929266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.929898 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.929902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:28.930006 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.930611 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:28.930628 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:28.933652 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":304,"threads_started":1}
W20260926 07:40:28.934091 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:28.925832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:51965 (local address 127.11.9.126:44565)
0926 07:40:28.925994 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:28.925999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:28.926463 (+ 464us) server_negotiation.cc:407] Connection header received
0926 07:40:28.926577 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:28.926581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:28.926685 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:28.926766 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:28.927481 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.928206 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.929384 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:28.929778 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:28.929877 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:28.933821 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:51965: BlockingRecv error: recv got EOF from 127.11.9.66:51965 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:29.410751 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:29.398519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:41821)
0926 07:40:29.399052 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:29.399067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:29.399221 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:29.399563 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:29.399567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:29.399992 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:29.400145 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.400153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.401159 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.401210 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.402114 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.402125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.402794 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.402799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.402906 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.407879 (+ 4973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:29.407897 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:29.410513 (+ 2616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":254,"threads_started":1}
W20260926 07:40:29.410897 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:29.398595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:41821 (local address 127.11.9.126:44565)
0926 07:40:29.398801 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:29.398811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:29.399153 (+ 342us) server_negotiation.cc:407] Connection header received
0926 07:40:29.399355 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:29.399361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:29.399428 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:29.399531 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:29.400308 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.401037 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.402235 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.402662 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.402757 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.410701 (+ 7944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:41821: BlockingRecv error: recv got EOF from 127.11.9.67:41821 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:29.490475 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:29.482630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:39089)
0926 07:40:29.483126 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:29.483142 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:29.483251 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:29.483529 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:29.483532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:29.483894 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:29.484041 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.484049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.484995 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.484999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.485906 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.485916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.486655 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.486661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.486838 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.487474 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:29.487491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:29.490247 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":258,"threads_started":1}
W20260926 07:40:29.490649 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:29.482779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:39089 (local address 127.11.9.126:44565)
0926 07:40:29.483010 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:29.483017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:29.483245 (+ 228us) server_negotiation.cc:407] Connection header received
0926 07:40:29.483351 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:29.483354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:29.483415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:29.483489 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:29.484161 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.484881 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.486031 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.486460 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.486616 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.490432 (+ 3816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:39089: BlockingRecv error: recv got EOF from 127.11.9.65:39089 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:29 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:29.943328 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:29.934875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:43247)
0926 07:40:29.935456 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:29.935476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:29.935649 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:29.936034 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:29.936041 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:29.936636 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:29.936812 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.936821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.938095 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.938099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.938994 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:29.939005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.939653 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.939657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:29.939769 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.940399 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:29.940416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:29.943108 (+ 2692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":293,"threads_started":1}
W20260926 07:40:29.943476 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:29.934986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:43247 (local address 127.11.9.126:44565)
0926 07:40:29.935179 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:29.935187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:29.935571 (+ 384us) server_negotiation.cc:407] Connection header received
0926 07:40:29.935782 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:29.935787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:29.935882 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:29.935992 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:29.936932 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.937968 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.939103 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:29.939513 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:29.939601 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:29.943288 (+ 3687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:43247: BlockingRecv error: recv got EOF from 127.11.9.66:43247 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:30.419895 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:30.411783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:53263)
0926 07:40:30.412296 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:30.412312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:30.412478 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:30.412808 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:30.412814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:30.413260 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:30.413423 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.413432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.414436 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.414443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.415321 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.415332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.416113 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.416119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.416288 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.416877 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:30.416893 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:30.419605 (+ 2712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":262,"threads_started":1}
W20260926 07:40:30.420114 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:30.411875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:53263 (local address 127.11.9.126:44565)
0926 07:40:30.412127 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:30.412135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:30.412403 (+ 268us) server_negotiation.cc:407] Connection header received
0926 07:40:30.412598 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:30.412603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:30.412674 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:30.412776 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:30.413569 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.414277 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.415473 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.415921 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.416061 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.419815 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:53263: BlockingRecv error: recv got EOF from 127.11.9.67:53263 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:30.499174 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:30.491491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:56969)
0926 07:40:30.492175 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:30.492191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:30.492298 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:30.492548 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:30.492551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:30.492923 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:30.493079 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.493088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.494057 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.494061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.495025 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.495036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.495670 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.495674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.495787 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.496290 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:30.496307 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:30.498939 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":417,"threads_started":1}
W20260926 07:40:30.499310 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:30.491529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:56969 (local address 127.11.9.126:44565)
0926 07:40:30.491804 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:30.491811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:30.492267 (+ 456us) server_negotiation.cc:407] Connection header received
0926 07:40:30.492390 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:30.492393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:30.492453 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:30.492529 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:30.493241 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.493950 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.495147 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.495548 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.495640 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.499110 (+ 3470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:56969: BlockingRecv error: recv got EOF from 127.11.9.65:56969 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:30 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:30.951831 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:30.944316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:38721)
0926 07:40:30.944870 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:30.944896 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:30.945002 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:30.945306 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:30.945309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:30.945686 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:30.945846 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.945854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.946798 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.946801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.947709 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:30.947719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.948338 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.948342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:30.948458 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.949024 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:30.949041 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:30.951594 (+ 2553us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":309,"threads_started":1}
W20260926 07:40:30.951990 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:30.944362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38721 (local address 127.11.9.126:44565)
0926 07:40:30.944540 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:30.944545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:30.944970 (+ 425us) server_negotiation.cc:407] Connection header received
0926 07:40:30.945097 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:30.945101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:30.945178 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:30.945255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:30.945976 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.946688 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.947831 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:30.948217 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:30.948303 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:30.951771 (+ 3468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:38721: BlockingRecv error: recv got EOF from 127.11.9.66:38721 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:31.429351 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:31.420982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:42039)
0926 07:40:31.421583 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:31.421599 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:31.421708 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:31.422002 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:31.422005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:31.422380 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:31.422589 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.422603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.423583 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.423587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.424530 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.424540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.425214 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.425222 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.425379 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.426039 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:31.426055 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:31.429045 (+ 2990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":351,"threads_started":1}
W20260926 07:40:31.429486 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:31.421046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:42039 (local address 127.11.9.126:44565)
0926 07:40:31.421299 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:31.421304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:31.421672 (+ 368us) server_negotiation.cc:407] Connection header received
0926 07:40:31.421799 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:31.421803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:31.421861 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:31.421957 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:31.422735 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.423465 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.424650 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.425037 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.425125 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.429277 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:42039: BlockingRecv error: recv got EOF from 127.11.9.67:42039 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:31.508116 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:31.500145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:40295)
0926 07:40:31.500663 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:31.500678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:31.500830 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:31.501207 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:31.501212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:31.501605 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:31.501741 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.501749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.502774 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.502782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.503719 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.503730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.504460 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.504466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.504637 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.505229 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:31.505246 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:31.507881 (+ 2635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":307,"threads_started":1}
W20260926 07:40:31.508272 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:31.500294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:40295 (local address 127.11.9.126:44565)
0926 07:40:31.500534 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:31.500542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:31.500764 (+ 222us) server_negotiation.cc:407] Connection header received
0926 07:40:31.500970 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:31.500982 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:31.501041 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:31.501146 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:31.501905 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.502631 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.503863 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.504303 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.504441 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.508045 (+ 3604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:40295: BlockingRecv error: recv got EOF from 127.11.9.65:40295 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:31 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:31.961229 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:31.952922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:50295)
0926 07:40:31.953471 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:31.953486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:31.953585 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:31.953857 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:31.953860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:31.954198 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:31.954351 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.954359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.955336 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.955340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.956605 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:31.956618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.957446 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.957452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:31.957617 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.958245 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:31.958262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:31.960934 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":334,"threads_started":1}
W20260926 07:40:31.961392 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:31.953002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:50295 (local address 127.11.9.126:44565)
0926 07:40:31.953212 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:31.953218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:31.953555 (+ 337us) server_negotiation.cc:407] Connection header received
0926 07:40:31.953675 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:31.953679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:31.953737 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:31.953818 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:31.954470 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.955211 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.956745 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:31.957254 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:31.957424 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:31.961115 (+ 3691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:50295: BlockingRecv error: recv got EOF from 127.11.9.66:50295 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:32.438437 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:32.430309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:52079)
0926 07:40:32.431037 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:32.431058 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:32.431184 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:32.431566 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:32.431571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:32.432093 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:32.432279 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.432288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.433282 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.433285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.434195 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.434206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.434827 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.434831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.434960 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.435510 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:32.435526 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:32.438224 (+ 2698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":467,"threads_started":1}
W20260926 07:40:32.438633 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:32.430484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:52079 (local address 127.11.9.126:44565)
0926 07:40:32.430708 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:32.430715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:32.431122 (+ 407us) server_negotiation.cc:407] Connection header received
0926 07:40:32.431329 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:32.431337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:32.431419 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:32.431497 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:32.432413 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.433115 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.434322 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.434704 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.434796 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.438393 (+ 3597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:52079: BlockingRecv error: recv got EOF from 127.11.9.67:52079 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 07:40:32.519405 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:32.509256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:51243 (local address 127.11.9.126:44565)
0926 07:40:32.509531 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:32.509541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:32.509788 (+ 247us) server_negotiation.cc:407] Connection header received
0926 07:40:32.509942 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:32.509950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:32.510076 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:32.510226 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:32.510989 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.512354 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.513523 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.514515 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.514653 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.519205 (+ 4552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:51243: BlockingRecv error: recv got EOF from 127.11.9.65:51243 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:40:32.519498 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:32.509044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:51243)
0926 07:40:32.509692 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:32.509709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:32.509819 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:32.510257 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:32.510260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:32.510651 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:32.510822 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.510831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.512483 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.512487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.513386 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.513397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.514699 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.514705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.515055 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.515800 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:32.515817 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:32.519044 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":395,"threads_started":1}
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:32 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:32.970623 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:32.962161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:41009)
0926 07:40:32.962604 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:32.962620 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:32.962741 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:32.963035 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:32.963038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:32.963404 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:32.963675 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.963690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.965053 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.965057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.966025 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:32.966037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.966713 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.966717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:32.966870 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.967527 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:32.967543 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:32.970262 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":252,"threads_started":1}
W20260926 07:40:32.970703 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:32.962248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:41009 (local address 127.11.9.126:44565)
0926 07:40:32.962491 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:32.962498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:32.962695 (+ 197us) server_negotiation.cc:407] Connection header received
0926 07:40:32.962838 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:32.962841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:32.962911 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:32.962991 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:32.963820 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.964925 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.966196 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:32.966591 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:32.966681 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:32.970473 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:41009: BlockingRecv error: recv got EOF from 127.11.9.66:41009 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:33.447522 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:33.439261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:36097)
0926 07:40:33.439739 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:33.439755 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:33.439913 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:33.440272 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:33.440276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:33.440690 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:33.440860 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.440868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.441951 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.441959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.442850 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.442861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.443584 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.443590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.443758 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.444330 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:33.444346 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:33.447179 (+ 2833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":256,"threads_started":1}
W20260926 07:40:33.447628 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:33.439358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:36097 (local address 127.11.9.126:44565)
0926 07:40:33.439586 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:33.439593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:33.439844 (+ 251us) server_negotiation.cc:407] Connection header received
0926 07:40:33.440050 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:33.440056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:33.440127 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:33.440237 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:33.441014 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.441813 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.443010 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.443435 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.443572 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.447360 (+ 3788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:36097: BlockingRecv error: recv got EOF from 127.11.9.67:36097 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:33.528653 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:33.520476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:38959)
0926 07:40:33.520992 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:33.521008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:33.521201 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:33.521590 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:33.521597 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:33.522027 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:33.522181 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.522190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.523179 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.523184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.524083 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.524094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.524822 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.524827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.525008 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.525701 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:33.525719 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:33.528426 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":287,"threads_started":1}
W20260926 07:40:33.528833 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:33.520527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38959 (local address 127.11.9.126:44565)
0926 07:40:33.520750 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:33.520758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:33.521089 (+ 331us) server_negotiation.cc:407] Connection header received
0926 07:40:33.521350 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:33.521354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:33.521423 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:33.521528 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:33.522322 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.523040 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.524225 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.524677 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.524811 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.528615 (+ 3804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:38959: BlockingRecv error: recv got EOF from 127.11.9.65:38959 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:33 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:33.981467 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:33.971647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:38667)
0926 07:40:33.972121 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:33.972136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:33.972245 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:33.972536 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:33.972539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:33.972898 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:33.973049 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.973058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.974072 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.974076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.975612 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:33.975627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.976717 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.976725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:33.976942 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.977666 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:33.977684 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:33.981160 (+ 3476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":264,"threads_started":1}
W20260926 07:40:33.981544 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:33.971664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38667 (local address 127.11.9.126:44565)
0926 07:40:33.971889 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:33.971899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:33.972220 (+ 321us) server_negotiation.cc:407] Connection header received
0926 07:40:33.972339 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:33.972343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:33.972406 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:33.972493 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:33.973213 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.973955 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.975784 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:33.976559 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:33.976704 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:33.981369 (+ 4665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:38667: BlockingRecv error: recv got EOF from 127.11.9.66:38667 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:34.461512 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:34.448730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:60993)
0926 07:40:34.449309 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:34.449326 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:34.449438 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:34.449715 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:34.449719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:34.450064 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:34.450259 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.450268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.451297 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.451303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.452668 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.452679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.453479 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.453483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.453607 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.458453 (+ 4846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:34.458471 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:34.461159 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":276,"threads_started":1}
W20260926 07:40:34.461575 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:34.448770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:60993 (local address 127.11.9.126:44565)
0926 07:40:34.448986 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:34.448992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:34.449394 (+ 402us) server_negotiation.cc:407] Connection header received
0926 07:40:34.449520 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:34.449524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:34.449584 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:34.449661 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:34.450382 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.451100 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.452854 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.453351 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.453441 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.461383 (+ 7942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:60993: BlockingRecv error: recv got EOF from 127.11.9.67:60993 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:34.538100 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:34.529647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:56465)
0926 07:40:34.530112 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:34.530127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:34.530240 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:34.530519 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:34.530522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:34.530904 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:34.531053 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.531061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.531978 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.531981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.532977 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.532988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.533933 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.533943 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.534082 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.534781 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:34.534832 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:34.537865 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":246,"threads_started":1}
W20260926 07:40:34.538369 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:34.529735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:56465 (local address 127.11.9.126:44565)
0926 07:40:34.529992 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:34.529999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:34.530203 (+ 204us) server_negotiation.cc:407] Connection header received
0926 07:40:34.530335 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:34.530339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:34.530401 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:34.530487 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:34.531170 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.531870 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.533129 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.533716 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.533856 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.538086 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:56465: BlockingRecv error: recv got EOF from 127.11.9.65:56465 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:34 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:34.990798 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:34.982390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:54885)
0926 07:40:34.982976 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:34.982992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:34.983147 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:34.983479 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:34.983483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:34.983898 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:34.984052 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.984060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.985102 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.985109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.986644 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:34.986660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.987367 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.987372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:34.987483 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.988010 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:34.988027 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:34.990588 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":322,"threads_started":1}
W20260926 07:40:34.990972 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:34.982594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:54885 (local address 127.11.9.126:44565)
0926 07:40:34.982809 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:34.982816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:34.983079 (+ 263us) server_negotiation.cc:407] Connection header received
0926 07:40:34.983276 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:34.983280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:34.983340 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:34.983454 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:34.984197 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.984970 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.986794 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:34.987215 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:34.987315 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:34.990765 (+ 3450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:54885: BlockingRecv error: recv got EOF from 127.11.9.66:54885 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:35.470136 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:35.462451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:37769)
0926 07:40:35.462968 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:35.462984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:35.463090 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:35.463364 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:35.463367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:35.463731 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:35.463886 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.463906 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.464815 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.464818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.465801 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.465812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.466548 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.466552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.466659 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.467232 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:35.467255 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:35.469902 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":270,"threads_started":1}
W20260926 07:40:35.470376 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:35.462522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:37769 (local address 127.11.9.126:44565)
0926 07:40:35.462684 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:35.462689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:35.463054 (+ 365us) server_negotiation.cc:407] Connection header received
0926 07:40:35.463181 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:35.463185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:35.463244 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:35.463321 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:35.464025 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.464705 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.465930 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.466405 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.466494 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.470127 (+ 3633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:37769: BlockingRecv error: recv got EOF from 127.11.9.67:37769 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:35.547518 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:35.538978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:44713)
0926 07:40:35.539423 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:35.539440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:35.539545 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:35.539839 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:35.539842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:35.540191 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:35.540374 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.540382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.541807 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.541814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.543100 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.543112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.543734 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.543737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.543847 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.544494 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:35.544511 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:35.547268 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":234,"threads_started":1}
W20260926 07:40:35.547652 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:35.539117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:44713 (local address 127.11.9.126:44565)
0926 07:40:35.539463 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:35.539469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:35.539491 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:40:35.539654 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:35.539658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:35.539719 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:35.539802 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:35.540516 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.541639 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.543247 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.543620 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.543709 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.547444 (+ 3735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:44713: BlockingRecv error: recv got EOF from 127.11.9.65:44713 (error 108)
Metrics: {"server-negotiator.queue_time_us":202}
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:35 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:35.999446 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:35.991688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:52753)
0926 07:40:35.992124 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:35.992140 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:35.992256 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:35.992540 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:35.992543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:35.992887 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:35.993075 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.993084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.994089 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.994093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.994969 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:35.994989 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.995639 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.995643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:35.995760 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.996386 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:35.996405 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:35.999219 (+ 2814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
W20260926 07:40:35.999626 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:35.991787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:52753 (local address 127.11.9.126:44565)
0926 07:40:35.992016 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:35.992023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:35.992221 (+ 198us) server_negotiation.cc:407] Connection header received
0926 07:40:35.992350 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:35.992353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:35.992423 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:35.992500 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:35.993244 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.993971 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.995093 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:35.995516 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:35.995614 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:35.999390 (+ 3776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:52753: BlockingRecv error: recv got EOF from 127.11.9.66:52753 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 07:40:36 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:36 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.67@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:36.478927 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:36.471184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.67:48709)
0926 07:40:36.471656 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:36.471671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:36.471795 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:36.472079 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:36.472082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:36.472427 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:36.472583 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:36.472591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.473691 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.473697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:36.474588 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:36.474598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.475242 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.475247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:36.475420 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:36.475990 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:36.476006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:36.478685 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":290,"threads_started":1}
W20260926 07:40:36.479118 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:36.471222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:48709 (local address 127.11.9.126:44565)
0926 07:40:36.471459 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:36.471468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:36.471766 (+ 298us) server_negotiation.cc:407] Connection header received
0926 07:40:36.471888 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:36.471892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:36.471951 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:36.472039 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:36.472734 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.473550 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.474724 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.475104 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.475233 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:36.478895 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.67:48709: BlockingRecv error: recv got EOF from 127.11.9.67:48709 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 07:40:36 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:36 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.65@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:36.556835 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:36.548667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.65:38595)
0926 07:40:36.549118 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:36.549134 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:36.549291 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:36.549612 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:36.549615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:36.550003 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:36.550186 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:36.550194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.551174 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.551178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:36.552073 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:36.552084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.552714 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.552718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:36.552823 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:36.553434 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:36.553450 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:36.556466 (+ 3016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":264,"threads_started":1}
W20260926 07:40:36.556910 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:36.548703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38595 (local address 127.11.9.126:44565)
0926 07:40:36.548943 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:36.548954 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:36.549248 (+ 294us) server_negotiation.cc:407] Connection header received
0926 07:40:36.549423 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:36.549427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:36.549488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:36.549570 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:36.550319 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.551054 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.552187 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:36.552602 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:36.552695 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:36.556654 (+ 3959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.65:38595: BlockingRecv error: recv got EOF from 127.11.9.65:38595 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 07:40:37 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 07:40:37 dist-test-slave-x492 krb5kdc[19546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.9.66@KRBTEST.COM for oryx/127.11.9.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 07:40:37.008773 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:37.000592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:44565 (local address 127.11.9.66:33333)
0926 07:40:37.001085 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:37.001100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:37.001277 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:37.001622 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:37.001627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:37.002045 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:37.002226 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:37.002234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.003234 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.003242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:37.004156 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:37.004168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.004959 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.004964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:37.005132 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:37.005806 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:37.005824 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:37.008505 (+ 2681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.9.126:44565: Server oryx/127.11.9.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":274,"threads_started":1}
W20260926 07:40:37.008949 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:37.000708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:33333 (local address 127.11.9.126:44565)
0926 07:40:37.000926 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:37.000934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:37.001212 (+ 278us) server_negotiation.cc:407] Connection header received
0926 07:40:37.001411 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:37.001423 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:37.001485 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:37.001599 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:37.002369 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.003074 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.004299 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.004807 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.004937 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:37.008697 (+ 3760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.9.66:33333: BlockingRecv error: recv got EOF from 127.11.9.66:33333 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:40:37.320818 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20133
I20260926 07:40:37.332708 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:37.321799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.11.9.65:44181)
0926 07:40:37.322393 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:37.322400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:37.322426 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:40:37.322483 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:37.322490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:37.322736 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:37.322892 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:37.324200 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.325336 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.326724 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.327271 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.327370 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:37.329769 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:37.329777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:37.329791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:37.329821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:37.332185 (+ 2364us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 07:40:37.332425 (+ 240us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60462: Request ticket server kudu/127.11.9.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":308,"threads_started":1}
W20260926 07:40:37.332811 20593 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60462: Request ticket server kudu/127.11.9.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 07:40:37.333014 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20133: Not authorized: Client connection negotiation failed: client connection to 127.11.9.65:44181: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 07:40:37.333256 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20133
I20260926 07:40:37.350592 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20284
I20260926 07:40:37.362360 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:37.351659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.11.9.66:41695)
0926 07:40:37.352220 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:37.352226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:37.352254 (+ 28us) server_negotiation.cc:407] Connection header received
0926 07:40:37.352310 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:37.352317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:37.352549 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:37.352746 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:37.354062 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.355114 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.356304 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.356923 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.357064 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:37.359514 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:37.359523 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:37.359536 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:37.359572 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:37.361736 (+ 2164us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 07:40:37.362027 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47920: Request ticket server kudu/127.11.9.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":279,"threads_started":1}
W20260926 07:40:37.362426 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20284: Not authorized: Client connection negotiation failed: client connection to 127.11.9.66:41695: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 07:40:37.362475 20594 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47920: Request ticket server kudu/127.11.9.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 07:40:37.362591 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20284
I20260926 07:40:37.378784 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20415
I20260926 07:40:37.390862 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:37.379782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.11.9.67:46589)
0926 07:40:37.380313 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:37.380320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:37.380344 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:40:37.380405 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:37.380411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:37.380623 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:37.380820 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:37.382144 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.383200 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.384401 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:37.384951 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:37.385096 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:37.387552 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:37.387561 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:37.387586 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:37.387622 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:37.390181 (+ 2559us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 07:40:37.390518 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43948: Request ticket server kudu/127.11.9.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":298,"threads_started":1}
W20260926 07:40:37.390949 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20415: Not authorized: Client connection negotiation failed: client connection to 127.11.9.67:46589: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.9.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 07:40:37.391000 20595 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43948: Request ticket server kudu/127.11.9.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 07:40:37.391175 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20415
I20260926 07:40:37.407332 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20064
I20260926 07:40:37.408588 20126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:37.522714 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20064
2026-09-26T07:40:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20972 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T07:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:37Z Disabled control of system clock
I20260926 07:40:37.596673 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39277
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:45621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:39277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 07:40:37.911993 20600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:37.912266 20600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:37.912379 20600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:37.919492 20600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:37.919600 20600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:37.919685 20600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:37.919780 20600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:37.919864 20600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:37.928596 20600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:39277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:39277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:37.929893 20600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:37.931592 20600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:37.943794 20605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:37.943810 20608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:37.943838 20606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:37.944519 20600 server_base.cc:1034] running on GCE node
I20260926 07:40:37.945478 20600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:37.946897 20600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:37.948127 20600 hybrid_clock.cc:648] HybridClock initialized: now 1790408437948092 us; error 55 us; skew 500 ppm
I20260926 07:40:37.948477 20600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:37.950811 20600 webserver.cc:460] Webserver started at http://127.11.9.126:40921/ using document root <none> and password file <none>
I20260926 07:40:37.951516 20600 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:37.951650 20600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:37.951968 20600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:37.954932 20600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "e0c97d0239b0488ebc07145b90e0a039"
format_stamp: "Formatted at 2026-09-26 07:40:37 on dist-test-slave-x492"
server_key: "2a4c53d0bb5a83bd6e0c4cb3f25f3d49"
server_key_iv: "ffbb6b4d6e9cd84fc07847ca009365a9"
server_key_version: "encryptionkey@0"
I20260926 07:40:37.955737 20600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "e0c97d0239b0488ebc07145b90e0a039"
format_stamp: "Formatted at 2026-09-26 07:40:37 on dist-test-slave-x492"
server_key: "2a4c53d0bb5a83bd6e0c4cb3f25f3d49"
server_key_iv: "ffbb6b4d6e9cd84fc07847ca009365a9"
server_key_version: "encryptionkey@0"
I20260926 07:40:37.960772 20600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 07:40:37.964018 20614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:37.965317 20600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:40:37.965492 20600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "e0c97d0239b0488ebc07145b90e0a039"
format_stamp: "Formatted at 2026-09-26 07:40:37 on dist-test-slave-x492"
server_key: "2a4c53d0bb5a83bd6e0c4cb3f25f3d49"
server_key_iv: "ffbb6b4d6e9cd84fc07847ca009365a9"
server_key_version: "encryptionkey@0"
I20260926 07:40:37.965711 20600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:37.989996 20600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:37.990852 20600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:37.991147 20600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:38.009231 20600 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:39277
I20260926 07:40:38.009320 20665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:39277 every 8 connection(s)
I20260926 07:40:38.010607 20600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:38.014405 20666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:38.014487 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20600
I20260926 07:40:38.014678 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:38.015300 11301 external_mini_cluster.cc:1428] Setting key 006679fa9170a99744266699d8751763
I20260926 07:40:38.028100 20666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039: Bootstrap starting.
I20260926 07:40:38.029922 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:38.017643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.11.9.126:39277)
0926 07:40:38.018774 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:38.018787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:38.018837 (+ 50us) server_negotiation.cc:407] Connection header received
0926 07:40:38.020576 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:38.020599 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:38.021094 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:38.021530 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:38.022223 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.023540 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.025486 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.026204 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.026346 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:38.027560 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:38.027568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:38.027581 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:38.028112 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:38.028200 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:38.028220 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:38.028354 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:38.028615 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 07:40:38.028946 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":410,"threads_started":1}
I20260926 07:40:38.031733 20666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:38.032899 20666 log.cc:824] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:38.035465 20666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039: No bootstrap required, opened a new log
I20260926 07:40:38.042580 20666 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } }
I20260926 07:40:38.042933 20666 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:38.043056 20666 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c97d0239b0488ebc07145b90e0a039, State: Initialized, Role: FOLLOWER
I20260926 07:40:38.043690 20666 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } }
I20260926 07:40:38.043957 20666 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:38.044096 20666 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:38.044271 20666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:38.046407 20666 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } }
I20260926 07:40:38.046841 20666 leader_election.cc:302] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c97d0239b0488ebc07145b90e0a039; no voters:
I20260926 07:40:38.047361 20666 leader_election.cc:288] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:38.047498 20671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:38.048192 20671 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [term 1 LEADER]: Becoming Leader. State: Replica: e0c97d0239b0488ebc07145b90e0a039, State: Running, Role: LEADER
I20260926 07:40:38.048851 20671 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } }
I20260926 07:40:38.049535 20666 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:38.051657 20673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c97d0239b0488ebc07145b90e0a039. Latest consensus state: current_term: 1 leader_uuid: "e0c97d0239b0488ebc07145b90e0a039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } } }
I20260926 07:40:38.051811 20672 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c97d0239b0488ebc07145b90e0a039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c97d0239b0488ebc07145b90e0a039" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 39277 } } }
I20260926 07:40:38.052043 20673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:38.052101 20672 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:38.054050 20679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:38.059259 20679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:38.067619 20679 catalog_manager.cc:1347] Generated new cluster ID: 49a3e65bde374a548b5eb2f4da5025b0
I20260926 07:40:38.067752 20679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:38.098054 20679 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:38.099923 20679 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:38.114887 20679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c97d0239b0488ebc07145b90e0a039: Generated new TSK 0
I20260926 07:40:38.115761 20679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:40:38.128656 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--builtin_ntp_servers=127.11.9.84:45621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 07:40:38.474571 20690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:38.474841 20690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:38.474992 20690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:38.482046 20690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:38.482160 20690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:38.482343 20690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:38.491012 20690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:38.492239 20690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:38.494150 20690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:38.506855 20696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:38.506870 20695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:38.506825 20698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:38.507668 20690 server_base.cc:1034] running on GCE node
I20260926 07:40:38.508262 20690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:38.509250 20690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:38.510474 20690 hybrid_clock.cc:648] HybridClock initialized: now 1790408438510439 us; error 53 us; skew 500 ppm
I20260926 07:40:38.510787 20690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:38.512882 20690 webserver.cc:460] Webserver started at http://127.11.9.65:34949/ using document root <none> and password file <none>
I20260926 07:40:38.513648 20690 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:38.513779 20690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:38.514110 20690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:38.516763 20690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "6fb23c1146744376b3125269d7da7e5b"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "af5617e8adba38df051ca52ca2631101"
server_key_iv: "7a97220c3ea588aa982f15e4f45b8c07"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.517647 20690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "6fb23c1146744376b3125269d7da7e5b"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "af5617e8adba38df051ca52ca2631101"
server_key_iv: "7a97220c3ea588aa982f15e4f45b8c07"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.522517 20690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:40:38.525731 20704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:38.526954 20690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260926 07:40:38.527143 20690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "6fb23c1146744376b3125269d7da7e5b"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "af5617e8adba38df051ca52ca2631101"
server_key_iv: "7a97220c3ea588aa982f15e4f45b8c07"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.527359 20690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:38.546857 20690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:38.547652 20690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:38.547920 20690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:38.548813 20690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:38.550410 20690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:38.550525 20690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:38.550642 20690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:38.550738 20690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:38.583226 20690 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:42111
I20260926 07:40:38.583313 20816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:42111 every 8 connection(s)
I20260926 07:40:38.584496 20690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:38.584813 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20690
I20260926 07:40:38.584993 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:38.585834 11301 external_mini_cluster.cc:1428] Setting key 857c3dc2879012f52f368f0688493b2b
I20260926 07:40:38.588862 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--builtin_ntp_servers=127.11.9.84:45621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 07:40:38.599057 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:38.587226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38627 (local address 127.11.9.126:39277)
0926 07:40:38.587720 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:38.587725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:38.591868 (+ 4143us) server_negotiation.cc:407] Connection header received
0926 07:40:38.591933 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:38.591940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:38.592033 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:38.592139 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:38.593200 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.594404 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.595933 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.596644 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.596796 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:38.598091 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:38.598100 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:38.598108 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:38.598132 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:38.598188 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:38.598207 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:38.598313 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:38.598517 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 07:40:38.598624 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":276,"threads_started":1}
I20260926 07:40:38.599795 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:38.587859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39277 (local address 127.11.9.65:38627)
0926 07:40:38.588896 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:38.588962 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:38.591533 (+ 2571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:38.592343 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:38.592351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:38.592435 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:38.592997 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:38.593013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.594553 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.594557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:38.595787 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:38.595798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:38.596846 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:38.596850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:38.597038 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:38.597789 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:38.597792 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:38.597829 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:38.597905 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:38.598313 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:38.598319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:38.598325 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 07:40:38.598474 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 07:40:38.598756 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":301,"threads_started":1}
I20260926 07:40:38.601392 20817 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39277
I20260926 07:40:38.601696 20817 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:38.602408 20817 heartbeater.cc:511] Master 127.11.9.126:39277 requested a full tablet report, sending...
I20260926 07:40:38.604369 20631 ts_manager.cc:194] Registered new tserver with Master: 6fb23c1146744376b3125269d7da7e5b (127.11.9.65:42111)
I20260926 07:40:38.605866 20631 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.65:38627
W20260926 07:40:38.907339 20821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:38.907595 20821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:38.907727 20821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:38.914933 20821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:38.915035 20821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:38.915163 20821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:38.924206 20821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:38.925454 20821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:38.927134 20821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:38.940412 20829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:38.940488 20827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:38.940452 20826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:38.941459 20821 server_base.cc:1034] running on GCE node
I20260926 07:40:38.942013 20821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:38.942914 20821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:38.944108 20821 hybrid_clock.cc:648] HybridClock initialized: now 1790408438944067 us; error 36 us; skew 500 ppm
I20260926 07:40:38.944455 20821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:38.946622 20821 webserver.cc:460] Webserver started at http://127.11.9.66:39539/ using document root <none> and password file <none>
I20260926 07:40:38.947294 20821 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:38.947405 20821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:38.947736 20821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:38.950591 20821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "9172e62344f145a58f16b7507752ece8"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "793dadefa2bd130143e4f0d5e16045a2"
server_key_iv: "5c7536a4a699e18ce9e7c23532aefed6"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.951375 20821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "9172e62344f145a58f16b7507752ece8"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "793dadefa2bd130143e4f0d5e16045a2"
server_key_iv: "5c7536a4a699e18ce9e7c23532aefed6"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.956382 20821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:40:38.959628 20835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:38.960806 20821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:38.960980 20821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "9172e62344f145a58f16b7507752ece8"
format_stamp: "Formatted at 2026-09-26 07:40:38 on dist-test-slave-x492"
server_key: "793dadefa2bd130143e4f0d5e16045a2"
server_key_iv: "5c7536a4a699e18ce9e7c23532aefed6"
server_key_version: "encryptionkey@0"
I20260926 07:40:38.961221 20821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:38.981685 20821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:38.982494 20821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:38.982784 20821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:38.983644 20821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:38.984992 20821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:38.985112 20821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:38.985271 20821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:38.985361 20821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:39.019277 20821 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:39949
I20260926 07:40:39.019392 20947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:39949 every 8 connection(s)
I20260926 07:40:39.020537 20821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:39.022334 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20821
I20260926 07:40:39.022504 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:39.022984 11301 external_mini_cluster.cc:1428] Setting key 531787c58897392b69cedaffcb4a6f88
I20260926 07:40:39.025960 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--builtin_ntp_servers=127.11.9.84:45621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 07:40:39.036345 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.024377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:59917 (local address 127.11.9.126:39277)
0926 07:40:39.024710 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.024715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.025797 (+ 1082us) server_negotiation.cc:407] Connection header received
0926 07:40:39.028546 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.028550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.028610 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.028693 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.030026 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.030814 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.033334 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.034039 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.034173 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.035462 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:39.035471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:39.035479 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:39.035503 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:39.035554 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:39.035572 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:39.035661 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:39.035855 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 07:40:39.035963 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260926 07:40:39.037148 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.023885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39277 (local address 127.11.9.66:59917)
0926 07:40:39.025633 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:39.025678 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:39.028266 (+ 2588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:39.028939 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:39.028952 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:39.029079 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:39.029859 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:39.029884 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.031020 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.031028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:39.033140 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:39.033157 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.034250 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.034255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:39.034456 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.035146 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:39.035149 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:39.035185 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:39.035260 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:39.035663 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:39.035670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:39.035675 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:40:39.035800 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 07:40:39.036091 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"spinlock_wait_cycles":27392,"thread_start_us":431,"threads_started":1}
I20260926 07:40:39.038637 20948 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39277
I20260926 07:40:39.038947 20948 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:39.039865 20948 heartbeater.cc:511] Master 127.11.9.126:39277 requested a full tablet report, sending...
I20260926 07:40:39.041333 20631 ts_manager.cc:194] Registered new tserver with Master: 9172e62344f145a58f16b7507752ece8 (127.11.9.66:39949)
I20260926 07:40:39.042104 20631 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.66:59917
W20260926 07:40:39.349870 20951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:39.350136 20951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:39.350301 20951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:39.357688 20951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:39.357774 20951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:39.357962 20951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:39.366811 20951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:39277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:39.368021 20951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:39.369834 20951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:39.382632 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:39.382609 20959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:39.382596 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:39.383426 20951 server_base.cc:1034] running on GCE node
I20260926 07:40:39.384053 20951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:39.385012 20951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:39.386333 20951 hybrid_clock.cc:648] HybridClock initialized: now 1790408439386260 us; error 59 us; skew 500 ppm
I20260926 07:40:39.386701 20951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:39.388911 20951 webserver.cc:460] Webserver started at http://127.11.9.67:38291/ using document root <none> and password file <none>
I20260926 07:40:39.389663 20951 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:39.389794 20951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:39.390121 20951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:39.392838 20951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "a4551588af654c41a1f6d0e382fd1e9a"
format_stamp: "Formatted at 2026-09-26 07:40:39 on dist-test-slave-x492"
server_key: "c9fb8d8885b54d76a14546f2525fa413"
server_key_iv: "ba2c1be439ca49623902ba234e9191d5"
server_key_version: "encryptionkey@0"
I20260926 07:40:39.393646 20951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "a4551588af654c41a1f6d0e382fd1e9a"
format_stamp: "Formatted at 2026-09-26 07:40:39 on dist-test-slave-x492"
server_key: "c9fb8d8885b54d76a14546f2525fa413"
server_key_iv: "ba2c1be439ca49623902ba234e9191d5"
server_key_version: "encryptionkey@0"
I20260926 07:40:39.398521 20951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:40:39.401721 20965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:39.402912 20951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 07:40:39.403091 20951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "a4551588af654c41a1f6d0e382fd1e9a"
format_stamp: "Formatted at 2026-09-26 07:40:39 on dist-test-slave-x492"
server_key: "c9fb8d8885b54d76a14546f2525fa413"
server_key_iv: "ba2c1be439ca49623902ba234e9191d5"
server_key_version: "encryptionkey@0"
I20260926 07:40:39.403297 20951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:39.420665 20951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:39.421495 20951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:39.421782 20951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:39.422677 20951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:39.424067 20951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:39.424194 20951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:39.424335 20951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:39.424428 20951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:39.457270 20951 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:46211
I20260926 07:40:39.457356 21077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:46211 every 8 connection(s)
I20260926 07:40:39.458582 20951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:39.460697 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 20951
I20260926 07:40:39.460856 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:39.461442 11301 external_mini_cluster.cc:1428] Setting key e3d1a7a2af9f675c8b6f6cd878758e39
I20260926 07:40:39.475060 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.461249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:58245 (local address 127.11.9.126:39277)
0926 07:40:39.461459 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.461465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.463517 (+ 2052us) server_negotiation.cc:407] Connection header received
0926 07:40:39.465314 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.465318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.465380 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.465454 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.466403 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.467109 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.468510 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.468952 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.469047 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.474358 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:39.474362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:39.474366 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:39.474381 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:39.474410 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:39.474421 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:39.474489 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:39.474692 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 07:40:39.474761 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:40:39.476270 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.461871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:39277 (local address 127.11.9.67:58245)
0926 07:40:39.463071 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:39.463126 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:39.464983 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:39.465659 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:39.465667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:39.465746 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:39.466269 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:39.466285 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.467229 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.467232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:39.468389 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:39.468403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.469119 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.469123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:39.469365 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.474090 (+ 4725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:39.474092 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:39.474128 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:39.474198 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:39.474511 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:39.474517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:39.474522 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:40:39.474688 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 07:40:39.475024 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":342,"threads_started":1}
I20260926 07:40:39.477530 21078 heartbeater.cc:347] Connected to a master server at 127.11.9.126:39277
I20260926 07:40:39.477768 21078 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:39.478430 21078 heartbeater.cc:511] Master 127.11.9.126:39277 requested a full tablet report, sending...
I20260926 07:40:39.479619 20631 ts_manager.cc:194] Registered new tserver with Master: a4551588af654c41a1f6d0e382fd1e9a (127.11.9.67:46211)
I20260926 07:40:39.480386 20631 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.67:58245
I20260926 07:40:39.488736 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 07:40:39.494383 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:39.492940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.11.9.126:39277)
0926 07:40:39.493185 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.493194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.493516 (+ 322us) server_negotiation.cc:407] Connection header received
0926 07:40:39.493650 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.493654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.493720 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.493796 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.494187 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40884: BlockingRecv error: recv got EOF from 127.0.0.1:40884 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:40:39.495146 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20690
I20260926 07:40:39.503283 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.496271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.11.9.65:42111)
0926 07:40:39.496777 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.496786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.496819 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:40:39.496871 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.496876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.497081 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.497282 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.497595 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.498668 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.500173 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.500714 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.500860 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.501874 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:39.501882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:39.501894 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:39.502488 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:39.502531 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:39.502552 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:39.502687 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:39.502828 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 07:40:39.502951 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":269,"threads_started":1}
I20260926 07:40:39.504469 20812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:39.585465 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20690
I20260926 07:40:39.602469 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20821
I20260926 07:40:39.610293 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.603293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.11.9.66:39949)
0926 07:40:39.603790 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.603806 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.603834 (+ 28us) server_negotiation.cc:407] Connection header received
0926 07:40:39.603893 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.603899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.604123 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.604292 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.604730 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.605757 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.607040 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.607637 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.607773 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.608784 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:39.608792 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:39.608819 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:39.609493 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:39.609541 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:39.609560 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:39.609683 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:39.609860 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 07:40:39.609985 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":232,"threads_started":1}
I20260926 07:40:39.611528 20943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:39.691594 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20821
I20260926 07:40:39.707286 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20951
I20260926 07:40:39.715174 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:39.708306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.11.9.67:46211)
0926 07:40:39.708972 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:39.708981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:39.709013 (+ 32us) server_negotiation.cc:407] Connection header received
0926 07:40:39.709070 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:39.709076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:39.709349 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:39.709500 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:39.709846 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.710943 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.712125 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:39.712700 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:39.712795 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:39.713814 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:39.713822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:39.713835 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:39.714403 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:39.714447 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:39.714464 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:39.714563 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:39.714680 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 07:40:39.714830 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":382,"threads_started":1}
I20260926 07:40:39.716389 21073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:39.797792 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20951
I20260926 07:40:39.813015 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 20600
I20260926 07:40:39.814253 20661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:39.913915 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 20600
2026-09-26T07:40:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2385 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T07:40:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:39Z Disabled control of system clock
I20260926 07:40:39.986123 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37105
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:37105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 07:40:40.302054 21097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:40.302335 21097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:40.302521 21097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:40.309661 21097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:40.309763 21097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:40.309868 21097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:40.309967 21097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:40.310048 21097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:40.319093 21097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:37105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:40.320354 21097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:40.322201 21097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:40.334156 21102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:40.334115 21103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:40.334197 21105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:40.335021 21097 server_base.cc:1034] running on GCE node
I20260926 07:40:40.335711 21097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:40.336973 21097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:40.338171 21097 hybrid_clock.cc:648] HybridClock initialized: now 1790408440338114 us; error 54 us; skew 500 ppm
I20260926 07:40:40.338515 21097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:40.340689 21097 webserver.cc:460] Webserver started at http://127.11.9.126:38201/ using document root <none> and password file <none>
I20260926 07:40:40.341459 21097 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:40.341578 21097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:40.341905 21097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:40.344755 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "7a391a7d05a641eb990d16b5b8b4c0f1"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "dce5ef89b337f1f8407fde95232030c3"
server_key_iv: "68be18d25ba122a004e85cbe3911db10"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.345602 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "7a391a7d05a641eb990d16b5b8b4c0f1"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "dce5ef89b337f1f8407fde95232030c3"
server_key_iv: "68be18d25ba122a004e85cbe3911db10"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.350493 21097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 07:40:40.353753 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:40.355082 21097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:40:40.355257 21097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "7a391a7d05a641eb990d16b5b8b4c0f1"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "dce5ef89b337f1f8407fde95232030c3"
server_key_iv: "68be18d25ba122a004e85cbe3911db10"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.355471 21097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:40.360764 21097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:40.361570 21097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:40.361857 21097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:40.378822 21097 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:37105
I20260926 07:40:40.378871 21162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:37105 every 8 connection(s)
I20260926 07:40:40.380208 21097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:40.383776 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:40.385584 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21097
I20260926 07:40:40.385859 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:40.386704 11301 external_mini_cluster.cc:1428] Setting key f6cfc5a3991ddbd26a55f4bf090a1ae9
I20260926 07:40:40.394906 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:40.388884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.11.9.126:37105)
0926 07:40:40.389903 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:40.389914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:40.389954 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:40:40.391438 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:40.391458 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:40.391878 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:40.392272 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:40.393350 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:40.393356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:40.393368 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:40.393383 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:40.393443 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:40.393458 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:40.393705 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:40.393832 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 07:40:40.394060 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":405,"threads_started":1}
I20260926 07:40:40.399159 21163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1: Bootstrap starting.
I20260926 07:40:40.402701 21163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:40.403762 21163 log.cc:824] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:40.406992 21163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1: No bootstrap required, opened a new log
I20260926 07:40:40.414444 21163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } }
I20260926 07:40:40.414925 21163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:40.415040 21163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a391a7d05a641eb990d16b5b8b4c0f1, State: Initialized, Role: FOLLOWER
I20260926 07:40:40.415625 21163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } }
I20260926 07:40:40.415817 21163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:40.415915 21163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:40.416029 21163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:40.418139 21163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } }
I20260926 07:40:40.418613 21163 leader_election.cc:302] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a391a7d05a641eb990d16b5b8b4c0f1; no voters:
I20260926 07:40:40.419171 21163 leader_election.cc:288] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:40.419309 21168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:40.420010 21168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [term 1 LEADER]: Becoming Leader. State: Replica: 7a391a7d05a641eb990d16b5b8b4c0f1, State: Running, Role: LEADER
I20260926 07:40:40.420513 21168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } }
I20260926 07:40:40.421411 21163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:40.423337 21170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a391a7d05a641eb990d16b5b8b4c0f1. Latest consensus state: current_term: 1 leader_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } } }
I20260926 07:40:40.423295 21169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a391a7d05a641eb990d16b5b8b4c0f1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37105 } } }
I20260926 07:40:40.423723 21169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:40.423727 21170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:40.424805 21174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:40.430927 21174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:40.440032 21174 catalog_manager.cc:1347] Generated new cluster ID: 0a7a3ceea5e84ad8901adbff715c8d5b
I20260926 07:40:40.440227 21174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:40.457716 21174 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:40.458163 21174 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:40.466885 21174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a391a7d05a641eb990d16b5b8b4c0f1: Generated new TSK 0
I20260926 07:40:40.467658 21174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:40:40.477916 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--builtin_ntp_servers=127.11.9.84:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 07:40:40.803586 21187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:40.803856 21187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:40.803944 21187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:40.811167 21187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:40.811264 21187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:40.811432 21187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:40.820451 21187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:40.821918 21187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:40.823805 21187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:40.836905 21192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:40.836841 21193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:40.836882 21195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:40.837805 21187 server_base.cc:1034] running on GCE node
I20260926 07:40:40.838348 21187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:40.839200 21187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:40.840389 21187 hybrid_clock.cc:648] HybridClock initialized: now 1790408440840339 us; error 45 us; skew 500 ppm
I20260926 07:40:40.840699 21187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:40.842952 21187 webserver.cc:460] Webserver started at http://127.11.9.65:44201/ using document root <none> and password file <none>
I20260926 07:40:40.843633 21187 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:40.843746 21187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:40.844156 21187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:40.848820 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "8c549e2a4a674cfaa7eb3d18d6af868f"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "7a985b4b9211a2ed1641bf642cc44e9a"
server_key_iv: "2b9d95445e99f474c4d5a40265d69e20"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.849641 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "8c549e2a4a674cfaa7eb3d18d6af868f"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "7a985b4b9211a2ed1641bf642cc44e9a"
server_key_iv: "2b9d95445e99f474c4d5a40265d69e20"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.854838 21187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:40:40.858237 21201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:40.859737 21187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:40:40.859902 21187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "8c549e2a4a674cfaa7eb3d18d6af868f"
format_stamp: "Formatted at 2026-09-26 07:40:40 on dist-test-slave-x492"
server_key: "7a985b4b9211a2ed1641bf642cc44e9a"
server_key_iv: "2b9d95445e99f474c4d5a40265d69e20"
server_key_version: "encryptionkey@0"
I20260926 07:40:40.860147 21187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:40.865888 21187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:40.866636 21187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:40.866895 21187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:40.867753 21187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:40.869053 21187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:40.869171 21187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:40.869381 21187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:40.869472 21187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:40.901854 21187 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:36427
I20260926 07:40:40.901908 21313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:36427 every 8 connection(s)
I20260926 07:40:40.903184 21187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:40.911337 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:40.905829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:51167 (local address 127.11.9.126:37105)
0926 07:40:40.906244 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:40.906248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:40.907460 (+ 1212us) server_negotiation.cc:407] Connection header received
0926 07:40:40.909129 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:40.909135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:40.909236 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:40.909356 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:40.910594 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:40.910600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:40.910605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:40.910622 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:40.910672 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:40.910683 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:40.910806 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:40.910993 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 07:40:40.911059 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":219,"threads_started":1}
I20260926 07:40:40.911593 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21187
I20260926 07:40:40.911810 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:40.912340 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:40.906424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37105 (local address 127.11.9.65:51167)
0926 07:40:40.907307 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:40.907342 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:40.908861 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:40.909608 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:40.909616 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:40.909706 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:40.910319 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:40.910321 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:40.910355 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:40.910434 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:40.910778 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:40.910788 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:40.910795 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 07:40:40.911072 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 07:40:40.911309 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":263,"threads_started":1}
I20260926 07:40:40.912441 11301 external_mini_cluster.cc:1428] Setting key 50b27161b83b88c73c6b954e06ee64b0
I20260926 07:40:40.913888 21314 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37105
I20260926 07:40:40.914199 21314 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:40.914783 21314 heartbeater.cc:511] Master 127.11.9.126:37105 requested a full tablet report, sending...
I20260926 07:40:40.915225 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--builtin_ntp_servers=127.11.9.84:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 07:40:40.916746 21128 ts_manager.cc:194] Registered new tserver with Master: 8c549e2a4a674cfaa7eb3d18d6af868f (127.11.9.65:36427)
W20260926 07:40:41.240666 21318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:41.240924 21318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:41.241118 21318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:41.248390 21318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:41.248486 21318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:41.248690 21318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:41.257498 21318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:41.258788 21318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:41.260541 21318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:41.273269 21323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:41.273386 21326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:41.273355 21324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:41.274168 21318 server_base.cc:1034] running on GCE node
I20260926 07:40:41.274750 21318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:41.275655 21318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:41.276873 21318 hybrid_clock.cc:648] HybridClock initialized: now 1790408441276834 us; error 56 us; skew 500 ppm
I20260926 07:40:41.277236 21318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:41.279351 21318 webserver.cc:460] Webserver started at http://127.11.9.66:40805/ using document root <none> and password file <none>
I20260926 07:40:41.280015 21318 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:41.280141 21318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:41.280443 21318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:41.283174 21318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "5d597bd83267499398662182231fbb9e"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "10b1e6fa4e7506ee805730da935f2bed"
server_key_iv: "c9b0a95ef89fc51f9f5089f0f2199d16"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.283931 21318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "5d597bd83267499398662182231fbb9e"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "10b1e6fa4e7506ee805730da935f2bed"
server_key_iv: "c9b0a95ef89fc51f9f5089f0f2199d16"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.288666 21318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260926 07:40:41.291828 21332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.293043 21318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:40:41.293246 21318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "5d597bd83267499398662182231fbb9e"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "10b1e6fa4e7506ee805730da935f2bed"
server_key_iv: "c9b0a95ef89fc51f9f5089f0f2199d16"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.293467 21318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:41.298890 21318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:41.299785 21318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:41.300051 21318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:41.300915 21318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:41.302356 21318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:41.302474 21318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.302618 21318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:41.302706 21318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.335505 21318 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:42509
I20260926 07:40:41.335553 21444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:42509 every 8 connection(s)
I20260926 07:40:41.336882 21318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:41.344884 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.339562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:50913 (local address 127.11.9.126:37105)
0926 07:40:41.339738 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:41.339742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:41.341137 (+ 1395us) server_negotiation.cc:407] Connection header received
0926 07:40:41.342866 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:41.342871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:41.342936 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:41.343028 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:41.344282 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:41.344286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:41.344288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:41.344300 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:41.344326 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:41.344336 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:41.344430 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:41.344619 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 07:40:41.344665 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:40:41.345805 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.340089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37105 (local address 127.11.9.66:50913)
0926 07:40:41.340991 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:41.341030 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:41.342600 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:41.343228 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:41.343235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:41.343324 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:41.344026 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:41.344029 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:41.344065 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:41.344146 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:41.344422 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:41.344429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:41.344434 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:40:41.344659 (+ 225us) client_negotiation.cc:240] Negotiation successful
0926 07:40:41.344844 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":273,"threads_started":1}
I20260926 07:40:41.346357 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21318
I20260926 07:40:41.346552 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:41.347133 11301 external_mini_cluster.cc:1428] Setting key 3a9bccd0645f2cc4aa7d1af0b97501c7
I20260926 07:40:41.347409 21445 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37105
I20260926 07:40:41.347716 21445 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:41.348340 21445 heartbeater.cc:511] Master 127.11.9.126:37105 requested a full tablet report, sending...
I20260926 07:40:41.349752 21128 ts_manager.cc:194] Registered new tserver with Master: 5d597bd83267499398662182231fbb9e (127.11.9.66:42509)
I20260926 07:40:41.349979 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--builtin_ntp_servers=127.11.9.84:45279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 07:40:41.669047 21448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:41.669392 21448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:41.669560 21448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:41.676443 21448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:41.676573 21448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:41.676753 21448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:41.685374 21448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:41.686610 21448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:41.688354 21448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:41.701066 21453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:41.701114 21456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:41.701139 21454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:41.701922 21448 server_base.cc:1034] running on GCE node
I20260926 07:40:41.702497 21448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:41.703366 21448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:41.704593 21448 hybrid_clock.cc:648] HybridClock initialized: now 1790408441704548 us; error 60 us; skew 500 ppm
I20260926 07:40:41.704895 21448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:40:41.707171 21448 webserver.cc:460] Webserver started at http://127.11.9.67:42201/ using document root <none> and password file <none>
I20260926 07:40:41.707821 21448 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:41.707947 21448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:41.708261 21448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:41.710948 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "fd8c50eb91ca4599937b2ee47105434a"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "5026d5acdd9fe43d404330ea7bef9c1a"
server_key_iv: "f0d86fe44f2053453fd7ea191daa55fd"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.711714 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "fd8c50eb91ca4599937b2ee47105434a"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "5026d5acdd9fe43d404330ea7bef9c1a"
server_key_iv: "f0d86fe44f2053453fd7ea191daa55fd"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.716444 21448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260926 07:40:41.719663 21462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.720990 21448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:40:41.721156 21448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "fd8c50eb91ca4599937b2ee47105434a"
format_stamp: "Formatted at 2026-09-26 07:40:41 on dist-test-slave-x492"
server_key: "5026d5acdd9fe43d404330ea7bef9c1a"
server_key_iv: "f0d86fe44f2053453fd7ea191daa55fd"
server_key_version: "encryptionkey@0"
I20260926 07:40:41.721419 21448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:41.726751 21448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:41.727483 21448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:41.727754 21448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:41.728595 21448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:41.729954 21448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:41.730080 21448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.730204 21448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:41.730291 21448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:41.761449 21448 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:37059
I20260926 07:40:41.761498 21574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:37059 every 8 connection(s)
I20260926 07:40:41.762796 21448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:41.770961 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.765489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:53859 (local address 127.11.9.126:37105)
0926 07:40:41.765661 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:41.765665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:41.767074 (+ 1409us) server_negotiation.cc:407] Connection header received
0926 07:40:41.768747 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:41.768753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:41.768819 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:41.768939 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:41.770221 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:41.770226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:41.770231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:41.770251 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:41.770286 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:41.770298 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:41.770427 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:41.770607 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 07:40:41.770674 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:40:41.771358 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21448
I20260926 07:40:41.771552 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:41.771924 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.766037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37105 (local address 127.11.9.67:53859)
0926 07:40:41.766917 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:41.766960 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:41.768480 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:41.769206 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:41.769224 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:41.769313 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:41.769916 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:40:41.769918 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:41.769954 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:40:41.770051 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:41.770388 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:41.770398 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:41.770407 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 07:40:41.770684 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 07:40:41.770924 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":277,"threads_started":1}
I20260926 07:40:41.771960 11301 external_mini_cluster.cc:1428] Setting key 7a0cff86f7b5ce176a691ac051c5b630
I20260926 07:40:41.773384 21575 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37105
I20260926 07:40:41.773695 21575 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:41.774309 21575 heartbeater.cc:511] Master 127.11.9.126:37105 requested a full tablet report, sending...
I20260926 07:40:41.775681 21128 ts_manager.cc:194] Registered new tserver with Master: fd8c50eb91ca4599937b2ee47105434a (127.11.9.67:37059)
I20260926 07:40:41.787176 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 07:40:41.793143 21316 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:40:41.791750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.11.9.126:37105)
0926 07:40:41.791934 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:41.791939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:41.792471 (+ 532us) server_negotiation.cc:407] Connection header received
0926 07:40:41.792564 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:41.792568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:41.792625 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:41.792704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:41.792963 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33062: BlockingRecv error: recv got EOF from 127.0.0.1:33062 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:40:41.793627 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21187
I20260926 07:40:41.797623 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.795002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.11.9.65:36427)
0926 07:40:41.795521 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:41.795527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:41.795552 (+ 25us) server_negotiation.cc:407] Connection header received
0926 07:40:41.795614 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:41.795619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:41.795817 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:41.795941 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:41.796809 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:41.796823 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:41.796839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:41.796859 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:41.796916 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:41.796937 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:41.797037 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:41.797197 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 07:40:41.797324 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260926 07:40:41.798853 21309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:41.878327 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21187
I20260926 07:40:41.894213 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21318
I20260926 07:40:41.898031 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:41.895118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.11.9.66:42509)
0926 07:40:41.895823 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:41.895829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:41.895855 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:40:41.895916 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:41.895922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:41.896134 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:41.896288 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:41.897291 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:41.897297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:41.897309 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:41.897327 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:41.897368 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:41.897383 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:41.897485 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:41.897644 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 07:40:41.897745 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":409,"threads_started":1}
I20260926 07:40:41.899189 21440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:41.981366 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21318
I20260926 07:40:41.999086 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21448
I20260926 07:40:42.002664 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:42.000014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.11.9.67:37059)
0926 07:40:42.000694 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:42.000701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:42.000731 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:40:42.000793 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:42.000799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:42.001017 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:42.001150 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:42.002056 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:42.002063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:42.002074 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:40:42.002085 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:42.002124 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:40:42.002137 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:42.002272 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:42.002318 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 07:40:42.002408 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":325,"threads_started":1}
I20260926 07:40:42.003980 21570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:42.087636 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21448
I20260926 07:40:42.105196 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21097
I20260926 07:40:42.106478 21158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:42.214728 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21097
2026-09-26T07:40:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2301 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:40:42 dist-test-slave-x492 krb5kdc[21593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:40:42 dist-test-slave-x492 krb5kdc[21593](info): set up 2 sockets
Sep 26 07:40:42 dist-test-slave-x492 krb5kdc[21593](info): commencing operation
krb5kdc: starting...
W20260926 07:40:43.314105 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:43 dist-test-slave-x492 krb5kdc[21593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:40:43.537007 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45769
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:43.855581 21606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:43.855881 21606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:43.856061 21606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:43.863207 21606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:40:43.863312 21606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:43.863364 21606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:43.863502 21606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:40:43.863582 21606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:40:43.873188 21606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:43.874459 21606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:43.875739 21606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:43.887293 21611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:43.887282 21612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:43.887341 21614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:43.888156 21606 server_base.cc:1034] running on GCE node
I20260926 07:40:43.888908 21606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:43.890357 21606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:43.891592 21606 hybrid_clock.cc:648] HybridClock initialized: now 1790408443891538 us; error 66 us; skew 500 ppm
I20260926 07:40:43.891961 21606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:43 dist-test-slave-x492 krb5kdc[21593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:43.895057 21606 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:40:43.896487 21606 webserver.cc:460] Webserver started at http://127.11.9.126:39171/ using document root <none> and password file <none>
I20260926 07:40:43.897233 21606 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:43.897374 21606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:43.897680 21606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:43.900496 21606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "5fddbcc810d84874802581352107d348"
format_stamp: "Formatted at 2026-09-26 07:40:43 on dist-test-slave-x492"
server_key: "08bfb88637f1bee68f0fbde181635536"
server_key_iv: "5ddfda704fdd1d34dc56aa6599964626"
server_key_version: "encryptionkey@0"
I20260926 07:40:43.901291 21606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "5fddbcc810d84874802581352107d348"
format_stamp: "Formatted at 2026-09-26 07:40:43 on dist-test-slave-x492"
server_key: "08bfb88637f1bee68f0fbde181635536"
server_key_iv: "5ddfda704fdd1d34dc56aa6599964626"
server_key_version: "encryptionkey@0"
I20260926 07:40:43.906180 21606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 07:40:43.909479 21621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:43.910742 21606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:43.910921 21606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "5fddbcc810d84874802581352107d348"
format_stamp: "Formatted at 2026-09-26 07:40:43 on dist-test-slave-x492"
server_key: "08bfb88637f1bee68f0fbde181635536"
server_key_iv: "5ddfda704fdd1d34dc56aa6599964626"
server_key_version: "encryptionkey@0"
I20260926 07:40:43.911108 21606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:43.945062 21606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:43.949031 21606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:43.949347 21606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:43.967295 21606 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45769
I20260926 07:40:43.967325 21672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45769 every 8 connection(s)
I20260926 07:40:43.968640 21606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:40:43.972196 21673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:43.978356 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21606
I20260926 07:40:43.978612 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:40:43.979400 11301 external_mini_cluster.cc:1428] Setting key 229592ac1ddb94cca52597cbab497f1c
I20260926 07:40:43.986454 21673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: Bootstrap starting.
I20260926 07:40:43.990064 21673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:43.991082 21673 log.cc:824] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:43.993919 21673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: No bootstrap required, opened a new log
Sep 26 07:40:43 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:44.004637 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:43.982221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39706 (local address 127.11.9.126:45769)
0926 07:40:43.983092 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:43.983108 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:43.983157 (+ 49us) server_negotiation.cc:407] Connection header received
0926 07:40:43.985490 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:43.985518 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:43.985994 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:43.986424 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:43.987748 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:43.989748 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:43.991010 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:43.992005 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:43.992153 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:43.996063 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:43.996073 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:43.996093 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:43.996141 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:44.000188 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:44.000844 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:44.000857 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:44.000866 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:44.001068 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:44.001451 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:44.001456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:44.001460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:44.002198 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:44.002529 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:44.002839 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 07:40:44.003285 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":356,"threads_started":1}
I20260926 07:40:44.004395 21673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } }
I20260926 07:40:44.004812 21673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:44.004921 21673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fddbcc810d84874802581352107d348, State: Initialized, Role: FOLLOWER
I20260926 07:40:44.005560 21673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } }
I20260926 07:40:44.005815 21673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:40:44.005961 21673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:40:44.006172 21673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:44.008737 21673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } }
I20260926 07:40:44.009344 21673 leader_election.cc:302] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fddbcc810d84874802581352107d348; no voters:
I20260926 07:40:44.010010 21673 leader_election.cc:288] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:40:44.010160 21678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:44.010874 21678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [term 1 LEADER]: Becoming Leader. State: Replica: 5fddbcc810d84874802581352107d348, State: Running, Role: LEADER
I20260926 07:40:44.011417 21678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } }
I20260926 07:40:44.012215 21673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:40:44.014317 21679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fddbcc810d84874802581352107d348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } } }
I20260926 07:40:44.014696 21679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:44.014415 21680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fddbcc810d84874802581352107d348. Latest consensus state: current_term: 1 leader_uuid: "5fddbcc810d84874802581352107d348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fddbcc810d84874802581352107d348" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45769 } } }
I20260926 07:40:44.014827 21680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348 [sys.catalog]: This master's current role is: LEADER
I20260926 07:40:44.015846 21683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:40:44.021871 21683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:40:44.030085 21683 catalog_manager.cc:1347] Generated new cluster ID: 15ae798e0d4943e7b5056429a8d701ee
I20260926 07:40:44.030194 21683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:40:44.040180 21683 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:40:44.041966 21683 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:40:44.059743 21683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: Generated new TSK 0
I20260926 07:40:44.060580 21683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:40:44.128249 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:44.496186 21699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:44.496460 21699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:44.496554 21699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:44.504388 21699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:44.504490 21699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:44.504694 21699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:40:44.514452 21699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:44.515890 21699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:44.517241 21699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:44.530236 21704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:44.530212 21705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:44.530447 21707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:44.530997 21699 server_base.cc:1034] running on GCE node
I20260926 07:40:44.531702 21699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:44.532680 21699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:44.533874 21699 hybrid_clock.cc:648] HybridClock initialized: now 1790408444533828 us; error 45 us; skew 500 ppm
I20260926 07:40:44.534194 21699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:44 dist-test-slave-x492 krb5kdc[21593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:44.543258 21699 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:40:44.544667 21699 webserver.cc:460] Webserver started at http://127.11.9.65:34217/ using document root <none> and password file <none>
I20260926 07:40:44.545447 21699 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:44.545549 21699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:44.545835 21699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:44.548672 21699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1"
format_stamp: "Formatted at 2026-09-26 07:40:44 on dist-test-slave-x492"
server_key: "d98dc907f397e4e67085bd2cf5016ed3"
server_key_iv: "3cf1461f0f62802ceff82ddcbb5acce2"
server_key_version: "encryptionkey@0"
I20260926 07:40:44.549491 21699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1"
format_stamp: "Formatted at 2026-09-26 07:40:44 on dist-test-slave-x492"
server_key: "d98dc907f397e4e67085bd2cf5016ed3"
server_key_iv: "3cf1461f0f62802ceff82ddcbb5acce2"
server_key_version: "encryptionkey@0"
I20260926 07:40:44.554772 21699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:40:44.558164 21714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:44.559589 21699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:40:44.559765 21699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1"
format_stamp: "Formatted at 2026-09-26 07:40:44 on dist-test-slave-x492"
server_key: "d98dc907f397e4e67085bd2cf5016ed3"
server_key_iv: "3cf1461f0f62802ceff82ddcbb5acce2"
server_key_version: "encryptionkey@0"
I20260926 07:40:44.560006 21699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:44.588709 21699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:44.592547 21699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:44.592792 21699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:44.593792 21699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:44.595024 21699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:44.595095 21699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:44.595185 21699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:44.595232 21699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:44.627866 21699 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:41899
I20260926 07:40:44.627912 21826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:41899 every 8 connection(s)
I20260926 07:40:44.629136 21699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:40:44.633948 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21699
I20260926 07:40:44.634161 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:40:44.634785 11301 external_mini_cluster.cc:1428] Setting key f3a7e32dd9bdcecc5aaf9706df2b44f9
Sep 26 07:40:44 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:44.650215 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:44.631686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:34625 (local address 127.11.9.126:45769)
0926 07:40:44.632128 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:44.632132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:44.632973 (+ 841us) server_negotiation.cc:407] Connection header received
0926 07:40:44.635008 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:44.635014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:44.635111 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:44.635241 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:44.637116 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:44.638334 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:44.639970 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:44.640668 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:44.640812 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:44.644523 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:44.644527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:44.644531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:44.644577 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:44.647243 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:44.647944 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:44.647952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:44.647955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:44.648073 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:44.648491 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:44.648497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:44.648501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:44.648747 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:44.648898 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:44.649631 (+ 733us) server_negotiation.cc:299] Negotiation successful
0926 07:40:44.649849 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":249,"threads_started":1}
I20260926 07:40:44.651150 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:44.632118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45769 (local address 127.11.9.65:34625)
0926 07:40:44.632806 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:44.632848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:44.634687 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:44.635491 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:44.635502 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:44.636172 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:44.636946 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:44.636965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:44.638568 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:44.638572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:44.639806 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:44.639818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:44.640873 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:44.640877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:44.641071 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:44.641865 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:44.641887 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:44.644305 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:44.647386 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:44.647394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:44.647412 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:44.647816 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:44.648182 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:44.648188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:44.648192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:44.648373 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:44.648876 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:44.648883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:44.649271 (+ 388us) client_negotiation.cc:769] Sending connection context
0926 07:40:44.649541 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 07:40:44.650059 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
I20260926 07:40:44.652783 21827 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45769
I20260926 07:40:44.653112 21827 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:44.653868 21827 heartbeater.cc:511] Master 127.11.9.126:45769 requested a full tablet report, sending...
I20260926 07:40:44.656255 21638 ts_manager.cc:194] Registered new tserver with Master: 3619ecfc2e4f4bbdb0b35ec33e33abb1 (127.11.9.65:41899)
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:40:44.657990 21638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:34625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:40:44.686261 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:45.005935 21833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:45.006214 21833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:45.006309 21833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:45.013783 21833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:45.013900 21833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:45.014015 21833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:40:45.023942 21833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:45.025157 21833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:45.026397 21833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:45.039403 21838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:45.039431 21841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:45.039520 21839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:45.040373 21833 server_base.cc:1034] running on GCE node
I20260926 07:40:45.041000 21833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:45.041930 21833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:45.043161 21833 hybrid_clock.cc:648] HybridClock initialized: now 1790408445043122 us; error 56 us; skew 500 ppm
I20260926 07:40:45.043488 21833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:45 dist-test-slave-x492 krb5kdc[21593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:45.046754 21833 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:40:45.048151 21833 webserver.cc:460] Webserver started at http://127.11.9.66:36257/ using document root <none> and password file <none>
I20260926 07:40:45.048841 21833 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:45.048985 21833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:45.049341 21833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:45.052109 21833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "f04b5b96ee8e43bab0d259e59127bef7"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "ac395bac315f28f98c885234b5348b58"
server_key_iv: "2a488cd93646d0d06ae3520084d18fe6"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.052850 21833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "f04b5b96ee8e43bab0d259e59127bef7"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "ac395bac315f28f98c885234b5348b58"
server_key_iv: "2a488cd93646d0d06ae3520084d18fe6"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.057821 21833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 07:40:45.061286 21848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.062603 21833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:40:45.062870 21833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "f04b5b96ee8e43bab0d259e59127bef7"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "ac395bac315f28f98c885234b5348b58"
server_key_iv: "2a488cd93646d0d06ae3520084d18fe6"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.063102 21833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:45.080045 21833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:45.084231 21833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:45.084517 21833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:45.085510 21833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:45.086815 21833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:45.086915 21833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.087088 21833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:45.087157 21833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.119940 21833 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:41187
I20260926 07:40:45.119989 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:41187 every 8 connection(s)
I20260926 07:40:45.121229 21833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:40:45.126967 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21833
I20260926 07:40:45.127219 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:40:45.127655 11301 external_mini_cluster.cc:1428] Setting key 861371861b7502d3a6a2781e9f1ea172
Sep 26 07:40:45 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:45.143498 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.123693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:56597 (local address 127.11.9.126:45769)
0926 07:40:45.123903 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.123910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.125607 (+ 1697us) server_negotiation.cc:407] Connection header received
0926 07:40:45.128689 (+ 3082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.128695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.128787 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.128898 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:45.130584 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.131302 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.132888 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.133382 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.133520 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.137248 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:45.137253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:45.137257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:45.137289 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:45.140256 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.141253 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.141257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.141259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.141387 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.141868 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.141871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.141873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.142126 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:45.142230 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.143107 (+ 877us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.143266 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:40:45.145113 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.124410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45769 (local address 127.11.9.66:56597)
0926 07:40:45.125454 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.125492 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.128407 (+ 2915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.129147 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.129155 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.129849 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:45.130418 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.130436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.131462 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.131467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.132730 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.132742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.133612 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.133619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.133853 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.134530 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:45.134556 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:45.136961 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:45.140413 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:45.140438 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:45.140459 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:45.141099 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:45.141499 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:45.141506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:45.141512 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:45.141760 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:45.142253 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:45.142263 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:45.142739 (+ 476us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.143036 (+ 297us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.143610 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":388,"threads_started":1}
I20260926 07:40:45.146733 21961 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45769
I20260926 07:40:45.147092 21961 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:45.147925 21961 heartbeater.cc:511] Master 127.11.9.126:45769 requested a full tablet report, sending...
I20260926 07:40:45.149519 21638 ts_manager.cc:194] Registered new tserver with Master: f04b5b96ee8e43bab0d259e59127bef7 (127.11.9.66:41187)
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:40:45.150759 21638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:56597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:40:45.178645 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:40:45.504192 21966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:40:45.504477 21966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:40:45.504681 21966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:40:45.511968 21966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:40:45.512073 21966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:40:45.512254 21966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:40:45.521971 21966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:40:45.523219 21966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:40:45.524500 21966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:40:45.537678 21971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:45.537745 21974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:40:45.537789 21972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:40:45.538249 21966 server_base.cc:1034] running on GCE node
I20260926 07:40:45.538961 21966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:40:45.539820 21966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:40:45.541018 21966 hybrid_clock.cc:648] HybridClock initialized: now 1790408445540980 us; error 30 us; skew 500 ppm
I20260926 07:40:45.541355 21966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:40:45 dist-test-slave-x492 krb5kdc[21593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:40:45.544384 21966 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:40:45.545778 21966 webserver.cc:460] Webserver started at http://127.11.9.67:45863/ using document root <none> and password file <none>
I20260926 07:40:45.546432 21966 fs_manager.cc:359] Metadata directory not provided
I20260926 07:40:45.546573 21966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:40:45.546855 21966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:40:45.549657 21966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "b7de492896d14fd2b5c9e8a2ea773d87"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "785093c270616a569d889d77ab5d6b13"
server_key_iv: "d2f0c29806483aa05dd0815a47f86928"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.550382 21966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "b7de492896d14fd2b5c9e8a2ea773d87"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "785093c270616a569d889d77ab5d6b13"
server_key_iv: "d2f0c29806483aa05dd0815a47f86928"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.555238 21966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:40:45.558532 21981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.559806 21966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 07:40:45.560051 21966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "b7de492896d14fd2b5c9e8a2ea773d87"
format_stamp: "Formatted at 2026-09-26 07:40:45 on dist-test-slave-x492"
server_key: "785093c270616a569d889d77ab5d6b13"
server_key_iv: "d2f0c29806483aa05dd0815a47f86928"
server_key_version: "encryptionkey@0"
I20260926 07:40:45.560245 21966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:40:45.591519 21966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:40:45.595847 21966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:40:45.596148 21966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:40:45.597083 21966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:40:45.598451 21966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:40:45.598552 21966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.598704 21966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:40:45.598778 21966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:40:45.635089 21966 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:40721
I20260926 07:40:45.635142 22093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:40721 every 8 connection(s)
I20260926 07:40:45.636261 21966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:40:45.640797 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 21966
I20260926 07:40:45.640993 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:40:45.641556 11301 external_mini_cluster.cc:1428] Setting key 527ab9e85a4b407cb7a2b75d81774139
Sep 26 07:40:45 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:40:45.655412 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.638676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:59787 (local address 127.11.9.126:45769)
0926 07:40:45.638878 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.638884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.639975 (+ 1091us) server_negotiation.cc:407] Connection header received
0926 07:40:45.642280 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.642285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.642371 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.642450 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:45.644261 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.645048 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.646588 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.646976 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.647069 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.650338 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:45.650342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:45.650345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:45.650374 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:45.652608 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.653326 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.653330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.653332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.653446 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.653816 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.653819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.653821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.654050 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:45.654159 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.654843 (+ 684us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.655047 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:40:45.656423 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.639156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45769 (local address 127.11.9.67:59787)
0926 07:40:45.639824 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.639859 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.642002 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.642724 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.642738 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.643498 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:40:45.644080 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.644119 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.645226 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.645233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.646470 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.646482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.647151 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.647155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.647331 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.647996 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:40:45.648017 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:40:45.650169 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:40:45.652743 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:45.652749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:45.652761 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:45.653158 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:45.653536 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:40:45.653539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:40:45.653541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:40:45.653721 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:40:45.654152 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:40:45.654158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:40:45.654452 (+ 294us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.654741 (+ 289us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.655246 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":281,"threads_started":1}
I20260926 07:40:45.657675 22094 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45769
I20260926 07:40:45.657969 22094 heartbeater.cc:464] Registering TS with master...
I20260926 07:40:45.658667 22094 heartbeater.cc:511] Master 127.11.9.126:45769 requested a full tablet report, sending...
I20260926 07:40:45.660061 21638 ts_manager.cc:194] Registered new tserver with Master: b7de492896d14fd2b5c9e8a2ea773d87 (127.11.9.67:40721)
I20260926 07:40:45.660941 21638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:59787
I20260926 07:40:45.661207 21827 heartbeater.cc:503] Master 127.11.9.126:45769 was elected leader, sending a full tablet report...
I20260926 07:40:45.668510 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:40:45.685124 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.672882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.11.9.126:45769)
0926 07:40:45.673033 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.673037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.673513 (+ 476us) server_negotiation.cc:407] Connection header received
0926 07:40:45.673633 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.673637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.673695 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.673772 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:45.675052 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.675747 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.676882 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.677342 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.677434 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.679681 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:45.679688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:45.679695 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:45.679732 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:45.682314 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.682932 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.682939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.682943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.683061 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.683433 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.683439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.683442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.683741 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:45.683993 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.684308 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.684508 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 07:40:45.688999 21638 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:40:45.695711 21638 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:40:45.718780 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.708629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35028 (local address 127.11.9.66:41187)
0926 07:40:45.709690 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.709703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.710373 (+ 670us) server_negotiation.cc:407] Connection header received
0926 07:40:45.710482 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.710491 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.710739 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.710976 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:45.711956 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.713584 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.716031 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.717569 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.717696 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.717900 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.718277 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.718460 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":657,"threads_started":1}
I20260926 07:40:45.718833 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.708468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:41187 (local address 127.0.0.1:35028)
0926 07:40:45.710122 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.710142 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.710257 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.711240 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.711251 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.711285 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:45.711735 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.711749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.713752 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.713757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.715883 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.715895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.717821 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.717830 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.718009 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.718084 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.718225 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.718434 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"mutex_wait_us":7,"thread_start_us":1413,"threads_started":1}
I20260926 07:40:45.719233 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.707987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.11.9.67:40721)
0926 07:40:45.711718 (+ 3731us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.711726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.711757 (+ 31us) server_negotiation.cc:407] Connection header received
0926 07:40:45.711846 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.711855 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.712123 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.712356 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:45.712833 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.714378 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.716940 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.718144 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.718342 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.718589 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.718743 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.718895 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3556,"thread_start_us":3390,"threads_started":1}
I20260926 07:40:45.719339 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.707785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:40721 (local address 127.0.0.1:58800)
0926 07:40:45.708574 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.708735 (+ 161us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.708971 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.712339 (+ 3368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.712346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.712371 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:45.712638 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.712645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.714540 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.714546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.716758 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.716772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.718394 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.718401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.718580 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.718617 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.718705 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.718875 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":43,"thread_start_us":374,"threads_started":1}
I20260926 07:40:45.722394 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.709022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41899 (local address 127.0.0.1:42802)
0926 07:40:45.712091 (+ 3069us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.712130 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.712259 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.713604 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.713608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.713629 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:45.714096 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.714107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.716120 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.716124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.719671 (+ 3547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.719683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.721612 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.721618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.721781 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.721818 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.721936 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.722053 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2919,"thread_start_us":2768,"threads_started":1}
I20260926 07:40:45.722453 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.709591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42802 (local address 127.11.9.65:41899)
0926 07:40:45.711214 (+ 1623us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.711223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.712219 (+ 996us) server_negotiation.cc:407] Connection header received
0926 07:40:45.712872 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.712880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.713147 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.713389 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:45.714257 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.715986 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.719893 (+ 3907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.721418 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.721568 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.721781 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.721982 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.722124 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":1276,"threads_started":1}
I20260926 07:40:45.723266 21896 tablet_service.cc:1467] Processing CreateTablet for tablet 29361d6f906d4a0c83987ad8551ce792 (DEFAULT_TABLE table=test-table [id=bccaf9eec7a4478d9192beae31e06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:45.723579 22029 tablet_service.cc:1467] Processing CreateTablet for tablet 29361d6f906d4a0c83987ad8551ce792 (DEFAULT_TABLE table=test-table [id=bccaf9eec7a4478d9192beae31e06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:45.724747 21896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29361d6f906d4a0c83987ad8551ce792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:45.724745 22029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29361d6f906d4a0c83987ad8551ce792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:45.725000 21762 tablet_service.cc:1467] Processing CreateTablet for tablet 29361d6f906d4a0c83987ad8551ce792 (DEFAULT_TABLE table=test-table [id=bccaf9eec7a4478d9192beae31e06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:40:45.726401 21762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29361d6f906d4a0c83987ad8551ce792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:40:45.736817 22112 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Bootstrap starting.
I20260926 07:40:45.739037 22113 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Bootstrap starting.
I20260926 07:40:45.740279 22114 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Bootstrap starting.
I20260926 07:40:45.741477 22112 tablet_bootstrap.cc:654] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:45.742777 22112 log.cc:824] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:45.743721 22114 tablet_bootstrap.cc:654] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:45.743805 22113 tablet_bootstrap.cc:654] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Neither blocks nor log segments found. Creating new log.
I20260926 07:40:45.744750 22113 log.cc:824] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:45.744889 22114 log.cc:824] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Log is configured to *not* fsync() on all Append() calls
I20260926 07:40:45.746055 22112 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: No bootstrap required, opened a new log
I20260926 07:40:45.746376 22112 ts_tablet_manager.cc:1397] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 07:40:45.747221 22113 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: No bootstrap required, opened a new log
I20260926 07:40:45.747468 22113 ts_tablet_manager.cc:1397] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260926 07:40:45.753947 22114 tablet_bootstrap.cc:492] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: No bootstrap required, opened a new log
I20260926 07:40:45.754209 22114 ts_tablet_manager.cc:1397] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260926 07:40:45.754863 22113 raft_consensus.cc:348] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.755218 22113 raft_consensus.cc:374] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:45.755321 22113 raft_consensus.cc:729] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f04b5b96ee8e43bab0d259e59127bef7, State: Initialized, Role: FOLLOWER
I20260926 07:40:45.755927 22113 consensus_queue.cc:260] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.757001 21961 heartbeater.cc:503] Master 127.11.9.126:45769 was elected leader, sending a full tablet report...
I20260926 07:40:45.757476 22113 ts_tablet_manager.cc:1428] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260926 07:40:45.758028 22112 raft_consensus.cc:348] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.758436 22112 raft_consensus.cc:374] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:45.758533 22112 raft_consensus.cc:729] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3619ecfc2e4f4bbdb0b35ec33e33abb1, State: Initialized, Role: FOLLOWER
I20260926 07:40:45.759073 22112 consensus_queue.cc:260] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.760645 22112 ts_tablet_manager.cc:1428] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260926 07:40:45.765825 22114 raft_consensus.cc:348] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.766167 22114 raft_consensus.cc:374] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:40:45.766276 22114 raft_consensus.cc:729] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7de492896d14fd2b5c9e8a2ea773d87, State: Initialized, Role: FOLLOWER
I20260926 07:40:45.766746 22114 consensus_queue.cc:260] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.767674 22094 heartbeater.cc:503] Master 127.11.9.126:45769 was elected leader, sending a full tablet report...
I20260926 07:40:45.768043 22114 ts_tablet_manager.cc:1428] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260926 07:40:45.791098 22120 raft_consensus.cc:482] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:45.791373 22120 raft_consensus.cc:504] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.793073 22120 leader_election.cc:288] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3619ecfc2e4f4bbdb0b35ec33e33abb1 (127.11.9.65:41899), f04b5b96ee8e43bab0d259e59127bef7 (127.11.9.66:41187)
I20260926 07:40:45.799706 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.793562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:45885 (local address 127.11.9.65:41899)
0926 07:40:45.793769 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.793777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.793815 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:40:45.793885 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.793889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.793966 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.794054 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:45.794460 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.795454 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.797791 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.798902 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.799048 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.799147 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.799340 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.799480 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:40:45.799711 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.793279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41899 (local address 127.11.9.67:45885)
0926 07:40:45.793560 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.793583 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.793707 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.794079 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.794084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.794116 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:45.794325 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.794338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.795598 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.795602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.797637 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.797652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.799064 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.799068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.799191 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.799214 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.799287 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.799372 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1}
I20260926 07:40:45.800341 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.793535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:60629 (local address 127.11.9.66:41187)
0926 07:40:45.794009 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.794017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.794240 (+ 223us) server_negotiation.cc:407] Connection header received
0926 07:40:45.794381 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.794385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.794478 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.794574 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:40:45.795020 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.795881 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.798135 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.799350 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.799475 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.799558 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.799975 (+ 417us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.800098 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260926 07:40:45.800370 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.793461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:41187 (local address 127.11.9.67:60629)
0926 07:40:45.794127 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 07:40:45.794143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:40:45.794263 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:40:45.794598 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:40:45.794600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:40:45.794619 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:40:45.794876 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.794882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.796049 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.796055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.797995 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:40:45.798004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.799561 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.799568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:40:45.799742 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.799774 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:40:45.799942 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 07:40:45.800072 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":321,"threads_started":1}
I20260926 07:40:45.800830 21782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29361d6f906d4a0c83987ad8551ce792" candidate_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" is_pre_election: true
I20260926 07:40:45.801306 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29361d6f906d4a0c83987ad8551ce792" candidate_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f04b5b96ee8e43bab0d259e59127bef7" is_pre_election: true
I20260926 07:40:45.801414 21782 raft_consensus.cc:2454] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7de492896d14fd2b5c9e8a2ea773d87 in term 0.
I20260926 07:40:45.801785 21916 raft_consensus.cc:2454] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7de492896d14fd2b5c9e8a2ea773d87 in term 0.
I20260926 07:40:45.802112 21983 leader_election.cc:302] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3619ecfc2e4f4bbdb0b35ec33e33abb1, b7de492896d14fd2b5c9e8a2ea773d87; no voters:
I20260926 07:40:45.802524 22120 raft_consensus.cc:2781] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:40:45.802687 22120 raft_consensus.cc:482] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:40:45.802795 22120 raft_consensus.cc:3037] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:45.804958 22120 raft_consensus.cc:504] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.805583 22120 leader_election.cc:288] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [CANDIDATE]: Term 1 election: Requested vote from peers 3619ecfc2e4f4bbdb0b35ec33e33abb1 (127.11.9.65:41899), f04b5b96ee8e43bab0d259e59127bef7 (127.11.9.66:41187)
I20260926 07:40:45.806094 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29361d6f906d4a0c83987ad8551ce792" candidate_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f04b5b96ee8e43bab0d259e59127bef7"
I20260926 07:40:45.806095 21782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29361d6f906d4a0c83987ad8551ce792" candidate_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1"
I20260926 07:40:45.806306 21916 raft_consensus.cc:3037] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:45.806306 21782 raft_consensus.cc:3037] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:40:45.809528 21782 raft_consensus.cc:2454] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7de492896d14fd2b5c9e8a2ea773d87 in term 1.
I20260926 07:40:45.809528 21916 raft_consensus.cc:2454] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7de492896d14fd2b5c9e8a2ea773d87 in term 1.
I20260926 07:40:45.809922 21983 leader_election.cc:302] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3619ecfc2e4f4bbdb0b35ec33e33abb1, b7de492896d14fd2b5c9e8a2ea773d87; no voters:
I20260926 07:40:45.810221 22120 raft_consensus.cc:2781] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:40:45.810667 22120 raft_consensus.cc:686] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 1 LEADER]: Becoming Leader. State: Replica: b7de492896d14fd2b5c9e8a2ea773d87, State: Running, Role: LEADER
I20260926 07:40:45.811129 22120 consensus_queue.cc:237] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } }
I20260926 07:40:45.815574 21638 catalog_manager.cc:5495] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7de492896d14fd2b5c9e8a2ea773d87 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7de492896d14fd2b5c9e8a2ea773d87" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 } health_report { overall_health: UNKNOWN } } }
I20260926 07:40:45.842849 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.837540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.11.9.67:40721)
0926 07:40:45.837747 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.837753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.837772 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:40:45.837983 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.837986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.838058 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.838223 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:40:45.838629 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.839352 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.840675 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.841069 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.841245 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.841738 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:40:45.842242 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:40:45.842401 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.842517 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.842597 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:40:45.850397 21916 raft_consensus.cc:1261] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 1 FOLLOWER]: Refusing update from remote peer b7de492896d14fd2b5c9e8a2ea773d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:45.850409 21782 raft_consensus.cc:1261] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 1 FOLLOWER]: Refusing update from remote peer b7de492896d14fd2b5c9e8a2ea773d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:40:45.851356 22122 consensus_queue.cc:1035] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f04b5b96ee8e43bab0d259e59127bef7" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 41187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:45.851691 22120 consensus_queue.cc:1035] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3619ecfc2e4f4bbdb0b35ec33e33abb1" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:40:45.861922 22125 mvcc.cc:205] Tried to move back new op lower bound from 7333512994193911808 to 7333512994045988864. Current Snapshot: MvccSnapshot[applied={T|T < 7333512994193911808}]
I20260926 07:40:45.866449 22127 mvcc.cc:205] Tried to move back new op lower bound from 7333512994193911808 to 7333512994045988864. Current Snapshot: MvccSnapshot[applied={T|T < 7333512994193911808}]
I20260926 07:40:45.878548 21637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:45.878883 21637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:40:45.882630 21637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5fddbcc810d84874802581352107d348: Sending DeleteTablet for 3 replicas of tablet 29361d6f906d4a0c83987ad8551ce792
I20260926 07:40:45.883790 22029 tablet_service.cc:1514] Processing DeleteTablet for tablet 29361d6f906d4a0c83987ad8551ce792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:45 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:58800
I20260926 07:40:45.884078 21762 tablet_service.cc:1514] Processing DeleteTablet for tablet 29361d6f906d4a0c83987ad8551ce792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:45 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:42802
I20260926 07:40:45.884397 21896 tablet_service.cc:1514] Processing DeleteTablet for tablet 29361d6f906d4a0c83987ad8551ce792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:40:45 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:35028
I20260926 07:40:45.885967 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21699
I20260926 07:40:45.897779 22136 tablet_replica.cc:330] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: stopping tablet replica
I20260926 07:40:45.898332 22136 raft_consensus.cc:2229] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:45.898964 22136 raft_consensus.cc:2258] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:45.899334 22135 tablet_replica.cc:330] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: stopping tablet replica
I20260926 07:40:45.899724 22137 tablet_replica.cc:330] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: stopping tablet replica
I20260926 07:40:45.899724 22135 raft_consensus.cc:2229] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:40:45.900403 22137 raft_consensus.cc:2229] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:40:45.900890 22137 raft_consensus.cc:2258] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:45.901033 22135 raft_consensus.cc:2258] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:40:45.901940 22136 ts_tablet_manager.cc:1905] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:45.902858 22137 ts_tablet_manager.cc:1905] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:45.908857 22135 ts_tablet_manager.cc:1905] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:40:45.909204 22136 ts_tablet_manager.cc:1918] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:45.909408 22136 log.cc:1196] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/29361d6f906d4a0c83987ad8551ce792
I20260926 07:40:45.909986 22136 ts_tablet_manager.cc:1939] T 29361d6f906d4a0c83987ad8551ce792 P 3619ecfc2e4f4bbdb0b35ec33e33abb1: Deleting consensus metadata
I20260926 07:40:45.910228 22137 ts_tablet_manager.cc:1918] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:45.910393 22137 log.cc:1196] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/29361d6f906d4a0c83987ad8551ce792
I20260926 07:40:45.910916 22137 ts_tablet_manager.cc:1939] T 29361d6f906d4a0c83987ad8551ce792 P f04b5b96ee8e43bab0d259e59127bef7: Deleting consensus metadata
I20260926 07:40:45.911962 21623 catalog_manager.cc:4841] TS 3619ecfc2e4f4bbdb0b35ec33e33abb1 (127.11.9.65:41899): tablet 29361d6f906d4a0c83987ad8551ce792 (table test-table [id=bccaf9eec7a4478d9192beae31e06917]) successfully deleted
I20260926 07:40:45.912731 21624 catalog_manager.cc:4841] TS f04b5b96ee8e43bab0d259e59127bef7 (127.11.9.66:41187): tablet 29361d6f906d4a0c83987ad8551ce792 (table test-table [id=bccaf9eec7a4478d9192beae31e06917]) successfully deleted
Sep 26 07:40:45 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:40:45.917325 22135 ts_tablet_manager.cc:1918] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:40:45.917510 22135 log.cc:1196] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/29361d6f906d4a0c83987ad8551ce792
I20260926 07:40:45.918057 22135 ts_tablet_manager.cc:1939] T 29361d6f906d4a0c83987ad8551ce792 P b7de492896d14fd2b5c9e8a2ea773d87: Deleting consensus metadata
I20260926 07:40:45.919672 21623 catalog_manager.cc:4841] TS b7de492896d14fd2b5c9e8a2ea773d87 (127.11.9.67:40721): tablet 29361d6f906d4a0c83987ad8551ce792 (table test-table [id=bccaf9eec7a4478d9192beae31e06917]) successfully deleted
I20260926 07:40:45.921876 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:45.887684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.11.9.65:41899)
0926 07:40:45.887902 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:45.887911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:45.900099 (+ 12188us) server_negotiation.cc:407] Connection header received
0926 07:40:45.900399 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:45.900406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:45.900483 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:45.900590 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:45.902304 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.903459 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.905283 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:45.905939 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:45.906029 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:45.915771 (+ 9742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:45.915782 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:45.915794 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:45.915825 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:45.918906 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.919718 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.919726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.919728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.919890 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:45.920269 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:45.920274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:45.920277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:45.920622 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:45.920923 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:45.921228 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 07:40:45.921425 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:40:45.922511 21822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:46.039861 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21699
I20260926 07:40:46.057862 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21833
Sep 26 07:40:46 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:40:46.072957 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:46.058902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.11.9.66:41187)
0926 07:40:46.059092 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:46.059097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:46.059111 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:40:46.059167 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:46.059171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:46.059229 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:46.059338 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:46.060514 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:46.061483 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:46.062713 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:46.063148 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:46.063271 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:46.067466 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:46.067479 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:46.067495 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:46.067528 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:46.070420 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:46.071027 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:46.071035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:46.071037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:46.071160 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:46.071505 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:46.071511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:46.071514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:46.071865 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:46.072140 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:46.072442 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 07:40:46.072653 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:40:46.073695 21956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:46.164469 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21833
I20260926 07:40:46.180958 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21966
Sep 26 07:40:46 dist-test-slave-x492 krb5kdc[21593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:40:46.198457 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:40:46.181772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.11.9.67:40721)
0926 07:40:46.181930 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 07:40:46.181935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:40:46.181996 (+ 61us) server_negotiation.cc:407] Connection header received
0926 07:40:46.182209 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:40:46.182213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:40:46.182270 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:40:46.182352 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:40:46.184148 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:46.184945 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:46.186889 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:40:46.187406 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:40:46.187496 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:40:46.192552 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:40:46.192563 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:40:46.192578 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:40:46.192617 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:40:46.195623 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:46.196272 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:46.196286 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:46.196288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:46.196429 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:40:46.196813 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:40:46.196818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:40:46.196823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:40:46.197242 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:40:46.197564 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0926 07:40:46.197852 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 07:40:46.198071 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:40:46.199091 22089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:46.319638 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21966
I20260926 07:40:46.338892 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 21606
I20260926 07:40:46.339926 21668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:40:46.446524 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 21606
2026-09-26T07:40:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4232 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:40:46 dist-test-slave-x492 krb5kdc[22145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:40:46 dist-test-slave-x492 krb5kdc[22145](info): set up 2 sockets
Sep 26 07:40:46 dist-test-slave-x492 krb5kdc[22145](info): commencing operation
krb5kdc: starting...
W20260926 07:40:47.550050 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:40:47 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:40:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:40:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangeradmin_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangeradmin_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangerlookup_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangerlookup_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
I20260926 07:40:47.854460 11301 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm0hQhn/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/postgres -l logfile start
2026-09-26 07:40:50.336 UTC [22175] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 07:40:50.336 UTC [22175] LOG: listening on IPv4 address "127.11.9.84", port 48733
2026-09-26 07:40:50.339 UTC [22175] LOG: listening on Unix socket "/tmp/.s.PGSQL.48733"
2026-09-26 07:40:50.360 UTC [22178] LOG: database system was shut down at 2026-09-26 07:40:48 UTC
2026-09-26 07:40:50.363 UTC [22175] LOG: database system is ready to accept connections
I20260926 07:40:51.329818 11301 mini_postgres.cc:96] Postgres bound to 48733
2026-09-26 07:40:51.346 UTC [22186] FATAL: database "slave" does not exist
127.11.9.84:48733 - accepting connections
I20260926 07:40:51.347442 11301 mini_ranger.cc:161] Starting Ranger...
I20260926 07:40:51.372646 11301 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 07:40:51.870771 11301 mini_ranger.cc:87] Created ranger Postgres database
I20260926 07:40:51.870909 11301 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin
2026-09-26 07:40:52,155 [I] DB FLAVOR :POSTGRES
2026-09-26 07:40:52,156 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 07:40:52,156 [I] Checking connection..
2026-09-26 07:40:52,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 07:40:52,407 [I] Checking connection passed.
2026-09-26 07:40:52,407 [I] --------- Verifying version history table ---------
2026-09-26 07:40:52,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:40:52,663 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 07:40:52,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:40:52,888 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 07:40:52,888 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 07:40:52,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 07:40:53.109 UTC [22250] WARNING: there is no transaction in progress
2026-09-26 07:40:53,115 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 07:40:53,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:40:53,348 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 07:40:53,348 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 07:40:53,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 07:40:53,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 07:40:53,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x492.c.gcp-upstream.internal','N') ;"
2026-09-26 07:40:54,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 07:40:54,294 [I] Table x_portal_user does not exist in database ranger
2026-09-26 07:40:54,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 07:40:54,545 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 07:40:54,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 07:40:54,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 07:40:55,004 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 07:40:55,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 07:40:55.921 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:55.937 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:55.954 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:55.970 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:55.988 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.009 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.073 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.079 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.094 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.104 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.115 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.129 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.139 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.146 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.153 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56.160 UTC [22358] WARNING: there is no transaction in progress
2026-09-26 07:40:56,798 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 07:40:56,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 07:40:57,041 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 07:40:57,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 07:40:57,300 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 07:40:57,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 07:40:57,549 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 07:40:57,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 07:40:57,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 07:40:58,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x492.c.gcp-upstream.internal';"
2026-09-26 07:40:58,238 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 07:40:58,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 07:40:58,503 [I] DB_PATCHES have already been applied
I20260926 07:40:58.509805 11301 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm0hQhn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 07:40:58.509896 11301 mini_ranger.cc:193] Using host: 127.11.9.84
I20260926 07:40:58.524984 11301 mini_ranger.cc:227] Ranger admin URL: http://127.11.9.84:46183
Sep 26, 2026 7:40:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:40:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:40:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 7:40:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 7:40:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 7:40:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 7:40:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 07:40:59.664400 11301 mini_ranger.cc:160] Time spent starting Ranger: real 8.317s user 0.000s sys 0.038s
Sep 26, 2026 7:41:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 7:41:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 7:41:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 7:41:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 7:41:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 07:41:21.048107 11301 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangerkms_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/rangerkms_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.9.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.9.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 07:41:21 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408481, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 07:41:21.237497 11301 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 07:41:21.257894 11301 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 07:41:22.123826 11301 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 07:41:22.124009 11301 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms
I20260926 07:41:22.124122 11301 mini_ranger_kms.cc:227] Using postgres at 127.11.9.84:48733
2026-09-26 07:41:22,544 [I] DB FLAVOR :POSTGRES
2026-09-26 07:41:22,544 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 07:41:22,544 [I] Checking connection
2026-09-26 07:41:22,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 07:41:22,791 [I] connection success
2026-09-26 07:41:22,792 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 07:41:22,792 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 07:41:22,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 07:41:23,024 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 07:41:23,025 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 07:41:23,025 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 07:41:23,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:48733/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 07:41:23,277 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 07:41:23.763459 11301 mini_ranger_kms.cc:310] Created kms service
I20260926 07:41:23.936122 11301 mini_ranger_kms.cc:326] Created kudu user
I20260926 07:41:23.986049 11301 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 07:41:23 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408481, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:41:24.395133 11301 mini_ranger_kms.cc:384] Added ranger policy
I20260926 07:41:24.395505 11301 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 07:41:24.395577 11301 mini_ranger_kms.cc:240] Using host: 127.11.9.84
I20260926 07:41:24.416668 11301 mini_ranger_kms.cc:276] Ranger KMS PID: 22538
I20260926 07:41:24.416857 11301 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.9.84:59849
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 7:41:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 07:41:26 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790408486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:41:26.789989 11301 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.553s user 0.112s sys 0.323s
I20260926 07:41:26.790268 11301 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 07:41:26.790468 11301 mini_ranger_kms.cc:401] 127.11.9.84:59849/kms/v1/keys
Sep 26 07:41:27 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:41:27 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:41:29 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:41:29.608457 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42587
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42587
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:59849/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:41:29.933107 22589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:41:29.933405 22589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:41:29.933490 22589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:41:29.940553 22589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:41:29.940653 22589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:41:29.940697 22589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:41:29.940755 22589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:41:29.940804 22589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:41:29.950492 22589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.9.84:59849/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42587
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:41:29.951637 22589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:41:29.952863 22589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:41:29.964550 22594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:29.964589 22595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:29.964553 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:41:29.965512 22589 server_base.cc:1034] running on GCE node
I20260926 07:41:29.966177 22589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:41:29.967445 22589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:41:29.968638 22589 hybrid_clock.cc:648] HybridClock initialized: now 1790408489968594 us; error 43 us; skew 500 ppm
I20260926 07:41:29.968981 22589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:41:29 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:41:29.978070 22589 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:41:29.980228 22589 webserver.cc:460] Webserver started at http://127.11.9.126:38083/ using document root <none> and password file <none>
I20260926 07:41:29.981101 22589 fs_manager.cc:359] Metadata directory not provided
I20260926 07:41:29.981284 22589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:41:29.981683 22589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:41:29 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:41:30.029999 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "93a53127e5384aceb3feb00022cb4561"
format_stamp: "Formatted at 2026-09-26 07:41:30 on dist-test-slave-x492"
server_key: "2c6ae58679c20a4474051bc325b8ac3d"
server_key_iv: "4f14792e16314f61b8a64e0160cb1ccb"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:30.030864 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "93a53127e5384aceb3feb00022cb4561"
format_stamp: "Formatted at 2026-09-26 07:41:30 on dist-test-slave-x492"
server_key: "2c6ae58679c20a4474051bc325b8ac3d"
server_key_iv: "4f14792e16314f61b8a64e0160cb1ccb"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:30.036644 22589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:41:30.055857 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:30.057480 22589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:41:30.057684 22589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "93a53127e5384aceb3feb00022cb4561"
format_stamp: "Formatted at 2026-09-26 07:41:30 on dist-test-slave-x492"
server_key: "2c6ae58679c20a4474051bc325b8ac3d"
server_key_iv: "4f14792e16314f61b8a64e0160cb1ccb"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:30.057920 22589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:41:30.077342 22589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:41:30.081027 22589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:41:30.081363 22589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:41:30.099771 22589 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42587
I20260926 07:41:30.099804 22655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42587 every 8 connection(s)
I20260926 07:41:30.101176 22589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:41:30.102870 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 22589
I20260926 07:41:30.103071 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:41:30.103696 11301 external_mini_cluster.cc:1428] Setting key 0640cfac53e8206e5e2f31e90f928617
I20260926 07:41:30.105324 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 07:41:30 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:41:30.118515 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: Bootstrap starting.
I20260926 07:41:30.122643 22656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: Neither blocks nor log segments found. Creating new log.
I20260926 07:41:30.123872 22656 log.cc:824] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: Log is configured to *not* fsync() on all Append() calls
I20260926 07:41:30.126823 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:30.105629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.11.9.126:42587)
0926 07:41:30.106381 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:30.106398 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:30.106448 (+ 50us) server_negotiation.cc:407] Connection header received
0926 07:41:30.108390 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:30.108416 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:30.108917 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:30.109367 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:30.110422 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:30.112520 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:30.113786 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:30.114789 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:30.114967 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:30.118756 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:30.118766 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:30.118784 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:30.118825 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:30.122501 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:30.123238 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:30.123247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:30.123255 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:30.123435 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:30.123783 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:30.123788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:30.123791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:30.124445 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:30.124690 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:30.125211 (+ 521us) server_negotiation.cc:299] Negotiation successful
0926 07:41:30.125584 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":301,"threads_started":1}
I20260926 07:41:30.127104 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: No bootstrap required, opened a new log
I20260926 07:41:30.134796 22656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } }
I20260926 07:41:30.135097 22656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:41:30.135167 22656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a53127e5384aceb3feb00022cb4561, State: Initialized, Role: FOLLOWER
I20260926 07:41:30.135623 22656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } }
I20260926 07:41:30.135808 22656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:41:30.135898 22656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:41:30.136019 22656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:41:30.138039 22656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } }
I20260926 07:41:30.138604 22656 leader_election.cc:302] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93a53127e5384aceb3feb00022cb4561; no voters:
I20260926 07:41:30.139253 22656 leader_election.cc:288] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:41:30.139377 22661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:41:30.140044 22661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [term 1 LEADER]: Becoming Leader. State: Replica: 93a53127e5384aceb3feb00022cb4561, State: Running, Role: LEADER
I20260926 07:41:30.140518 22661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } }
I20260926 07:41:30.141216 22656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:41:30.143033 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93a53127e5384aceb3feb00022cb4561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } } }
I20260926 07:41:30.143352 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a53127e5384aceb3feb00022cb4561. Latest consensus state: current_term: 1 leader_uuid: "93a53127e5384aceb3feb00022cb4561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a53127e5384aceb3feb00022cb4561" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42587 } } }
I20260926 07:41:30.143548 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [sys.catalog]: This master's current role is: LEADER
I20260926 07:41:30.143636 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561 [sys.catalog]: This master's current role is: LEADER
I20260926 07:41:30.144359 22667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:41:30.155277 22667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:41:30.155988 22656 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 07:41:30.173864 22667 catalog_manager.cc:1347] Generated new cluster ID: 6dde2de94bba4003849ee39ba8f269ac
I20260926 07:41:30.174005 22667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:41:30.187184 22667 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:41:30.189038 22667 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:41:30.199318 22667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: Generated new TSK 0
I20260926 07:41:30.200340 22667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:41:30.905663 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:59849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:41:31 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 07:41:31.374521 22701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:41:31.374888 22701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:41:31.375102 22701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:41:31.386713 22701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:41:31.386868 22701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:41:31.387068 22701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:41:31.402648 22701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.9.84:59849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:41:31.404587 22701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:41:31.406322 22701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:41:31.422654 22711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:31.424121 22712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:41:31.432533 22701 server_base.cc:1034] running on GCE node
W20260926 07:41:31.433211 22714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:41:31.434062 22701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:41:31.435547 22701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:41:31.436848 22701 hybrid_clock.cc:648] HybridClock initialized: now 1790408491436788 us; error 49 us; skew 500 ppm
I20260926 07:41:31.437288 22701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:41:31 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:41:31.441802 22701 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:41:31.453540 22701 webserver.cc:460] Webserver started at http://127.11.9.65:33943/ using document root <none> and password file <none>
I20260926 07:41:31.454527 22701 fs_manager.cc:359] Metadata directory not provided
I20260926 07:41:31.454679 22701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:41:31.455039 22701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:41:31 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:41:31.477317 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "caafd54a97b7463fb1406793207c9a5b"
format_stamp: "Formatted at 2026-09-26 07:41:31 on dist-test-slave-x492"
server_key: "b7f07608e582344a5bd91e0adf3aafe5"
server_key_iv: "905f50f412ec20e7675149e791912025"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:31.479892 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "caafd54a97b7463fb1406793207c9a5b"
format_stamp: "Formatted at 2026-09-26 07:41:31 on dist-test-slave-x492"
server_key: "b7f07608e582344a5bd91e0adf3aafe5"
server_key_iv: "905f50f412ec20e7675149e791912025"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:31.496598 22701 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.007s sys 0.008s
I20260926 07:41:31.521193 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:31.523212 22701 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.005s
I20260926 07:41:31.523398 22701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "caafd54a97b7463fb1406793207c9a5b"
format_stamp: "Formatted at 2026-09-26 07:41:31 on dist-test-slave-x492"
server_key: "b7f07608e582344a5bd91e0adf3aafe5"
server_key_iv: "905f50f412ec20e7675149e791912025"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:31.523658 22701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:41:31.547601 22701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:41:31.552695 22701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:41:31.553093 22701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:41:31.555151 22701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:41:31.558147 22701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:41:31.558291 22701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 07:41:31.558490 22701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:41:31.558598 22701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:31.635336 22701 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44543
I20260926 07:41:31.637007 22701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:41:31.635354 22833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44543 every 8 connection(s)
I20260926 07:41:31.637897 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 22701
I20260926 07:41:31.638144 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:41:31.639206 11301 external_mini_cluster.cc:1428] Setting key 9dda5c22cfa81e6071f33420f51085cf
Sep 26 07:41:31 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:41:31.702942 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:31.642522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:38107 (local address 127.11.9.126:42587)
0926 07:41:31.677444 (+ 34922us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:31.677451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:31.677478 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:41:31.680240 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:31.680247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:31.680361 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:31.680442 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:31.683752 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:31.684981 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:31.687523 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:31.688283 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:31.688380 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:31.693909 (+ 5529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:31.693917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:31.693921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:31.693971 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:31.697522 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:31.700111 (+ 2589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:31.700117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:31.700121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:31.700290 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:31.700830 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:31.700836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:31.700839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:31.701246 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:31.701393 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:31.702334 (+ 941us) server_negotiation.cc:299] Negotiation successful
0926 07:41:31.702556 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34798,"thread_start_us":34950,"threads_started":1}
I20260926 07:41:31.704403 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:31.640994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42587 (local address 127.11.9.65:38107)
0926 07:41:31.673931 (+ 32937us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:31.673995 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:31.678461 (+ 4466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:31.681016 (+ 2555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:31.681028 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:31.681719 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:41:31.682583 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:31.682608 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:31.685231 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:31.685237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:31.687355 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:31.687370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:31.688723 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:31.688729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:31.689023 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:31.690037 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:41:31.690067 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:41:31.693606 (+ 3539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:41:31.699316 (+ 5710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:31.699326 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:31.699346 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:31.699940 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:31.700430 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:31.700435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:31.700438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:31.700674 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:31.701394 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:41:31.701402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:41:31.701849 (+ 447us) client_negotiation.cc:769] Sending connection context
0926 07:41:31.702137 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 07:41:31.702673 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32529,"thread_start_us":35033,"threads_started":1}
I20260926 07:41:31.706372 22834 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42587
I20260926 07:41:31.706676 22834 heartbeater.cc:464] Registering TS with master...
I20260926 07:41:31.707508 22834 heartbeater.cc:511] Master 127.11.9.126:42587 requested a full tablet report, sending...
I20260926 07:41:31.709919 22621 ts_manager.cc:194] Registered new tserver with Master: caafd54a97b7463fb1406793207c9a5b (127.11.9.65:44543)
I20260926 07:41:31.711934 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:38107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:41:31.738313 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:59849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:41:31 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
W20260926 07:41:32.174260 22840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:41:32.174651 22840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:41:32.174814 22840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:41:32.184698 22840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:41:32.184897 22840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:41:32.185127 22840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:41:32.200551 22840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.9.84:59849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:41:32.202461 22840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:41:32.204485 22840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:41:32.248530 22847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:32.251113 22848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:32.255697 22851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:41:32.256209 22840 server_base.cc:1034] running on GCE node
I20260926 07:41:32.257195 22840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:41:32.261436 22840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:41:32.262744 22840 hybrid_clock.cc:648] HybridClock initialized: now 1790408492262599 us; error 127 us; skew 500 ppm
I20260926 07:41:32.263144 22840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:41:32.267405 22840 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:41:32.269657 22840 webserver.cc:460] Webserver started at http://127.11.9.66:45075/ using document root <none> and password file <none>
I20260926 07:41:32.270654 22840 fs_manager.cc:359] Metadata directory not provided
I20260926 07:41:32.270834 22840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:41:32.271204 22840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:41:32.290560 22840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "165d4fbca3e34f6cab5449c12e022310"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "a7ba13939f705b13d13efcc013c4c738"
server_key_iv: "67643d78d0607d0fa84098c5a3c4792d"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.291692 22840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "165d4fbca3e34f6cab5449c12e022310"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "a7ba13939f705b13d13efcc013c4c738"
server_key_iv: "67643d78d0607d0fa84098c5a3c4792d"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.299057 22840 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260926 07:41:32.328431 22857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.334693 22840 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.002s sys 0.002s
I20260926 07:41:32.334990 22840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "165d4fbca3e34f6cab5449c12e022310"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "a7ba13939f705b13d13efcc013c4c738"
server_key_iv: "67643d78d0607d0fa84098c5a3c4792d"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.335359 22840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:41:32.367262 22840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:41:32.372520 22840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:41:32.372897 22840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:41:32.381119 22840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:41:32.389467 22840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:41:32.389607 22840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.389758 22840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:41:32.389837 22840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.433910 22840 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:43175
I20260926 07:41:32.433918 22976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:43175 every 8 connection(s)
I20260926 07:41:32.435316 22840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:41:32.436558 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 22840
I20260926 07:41:32.436739 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:41:32.437300 11301 external_mini_cluster.cc:1428] Setting key 8d9039b9b55a7139fb14d6ea39eeed12
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:41:32.457257 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:32.438507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:56215 (local address 127.11.9.126:42587)
0926 07:41:32.440765 (+ 2258us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:32.440770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:32.440789 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:41:32.442839 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:32.442844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:32.442924 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:32.443043 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:32.444490 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:32.445336 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:32.447049 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:32.447451 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:32.447589 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:32.451474 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:32.451478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:32.451482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:32.451511 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:32.454134 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:32.454898 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:32.454904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:32.454908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:32.455059 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:32.455501 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:32.455508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:32.455511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:32.455782 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:32.455935 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:32.456602 (+ 667us) server_negotiation.cc:299] Negotiation successful
0926 07:41:32.456822 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2143,"thread_start_us":2017,"threads_started":1}
I20260926 07:41:32.458293 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:32.439207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42587 (local address 127.11.9.66:56215)
0926 07:41:32.440035 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:32.440086 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:32.442576 (+ 2490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:32.443341 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:32.443349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:32.443767 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:41:32.444320 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:32.444336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:32.445495 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:32.445500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:32.446901 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:32.446919 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:32.447669 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:32.447675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:32.447905 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:32.448585 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:41:32.448607 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:41:32.451265 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:41:32.454284 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:32.454292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:32.454309 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:32.454784 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:32.455188 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:32.455194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:32.455197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:32.455391 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:32.455913 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:41:32.455921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:41:32.456244 (+ 323us) client_negotiation.cc:769] Sending connection context
0926 07:41:32.456519 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 07:41:32.457043 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":375,"threads_started":1}
I20260926 07:41:32.459697 22977 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42587
I20260926 07:41:32.460011 22977 heartbeater.cc:464] Registering TS with master...
I20260926 07:41:32.460690 22977 heartbeater.cc:511] Master 127.11.9.126:42587 requested a full tablet report, sending...
I20260926 07:41:32.462379 22621 ts_manager.cc:194] Registered new tserver with Master: 165d4fbca3e34f6cab5449c12e022310 (127.11.9.66:43175)
I20260926 07:41:32.463219 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:56215
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:41:32.500110 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:59849/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:41:32.715525 22834 heartbeater.cc:503] Master 127.11.9.126:42587 was elected leader, sending a full tablet report...
W20260926 07:41:32.840842 22983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:41:32.841202 22983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:41:32.841380 22983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:41:32.848727 22983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:41:32.848820 22983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:41:32.848928 22983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:41:32.858484 22983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.9.84:59849/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:41:32.859690 22983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:41:32.860968 22983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:41:32.874246 22988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:32.874279 22989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:41:32.874246 22991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:41:32.875098 22983 server_base.cc:1034] running on GCE node
I20260926 07:41:32.875690 22983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:41:32.876586 22983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:41:32.877805 22983 hybrid_clock.cc:648] HybridClock initialized: now 1790408492877762 us; error 38 us; skew 500 ppm
I20260926 07:41:32.878118 22983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:41:32.881079 22983 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:41:32.882436 22983 webserver.cc:460] Webserver started at http://127.11.9.67:44555/ using document root <none> and password file <none>
I20260926 07:41:32.883095 22983 fs_manager.cc:359] Metadata directory not provided
I20260926 07:41:32.883244 22983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:41:32.883538 22983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:41:32 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:41:32.898792 22983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "38ce1537de2e4e67833b1cb103b18f55"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "e1a2603182429a3522b8d189392bda20"
server_key_iv: "bc873b00ac6b32e91ec9d42c6d06b608"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.899840 22983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "38ce1537de2e4e67833b1cb103b18f55"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "e1a2603182429a3522b8d189392bda20"
server_key_iv: "bc873b00ac6b32e91ec9d42c6d06b608"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.906565 22983 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 07:41:32.923128 22998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.924670 22983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:41:32.924877 22983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "38ce1537de2e4e67833b1cb103b18f55"
format_stamp: "Formatted at 2026-09-26 07:41:32 on dist-test-slave-x492"
server_key: "e1a2603182429a3522b8d189392bda20"
server_key_iv: "bc873b00ac6b32e91ec9d42c6d06b608"
server_key_version: "kuduclusterkey@0"
I20260926 07:41:32.925107 22983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:41:32.951869 22983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:41:32.955950 22983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:41:32.956265 22983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:41:32.957269 22983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:41:32.958606 22983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:41:32.958711 22983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.958858 22983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:41:32.958931 22983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:41:32.991863 22983 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:44119
I20260926 07:41:32.991892 23110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:44119 every 8 connection(s)
I20260926 07:41:32.993249 22983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:41:32.994611 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 22983
I20260926 07:41:32.994824 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:41:32.995313 11301 external_mini_cluster.cc:1428] Setting key cb884a1ba868b01f0892fba31301f00a
Sep 26 07:41:33 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408492, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:41:33.012581 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:32.996937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:33907 (local address 127.11.9.126:42587)
0926 07:41:32.997444 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:32.997450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:32.997923 (+ 473us) server_negotiation.cc:407] Connection header received
0926 07:41:32.999827 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:32.999831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:32.999907 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:32.999994 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:33.001446 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.002156 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.003592 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.003995 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.004085 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.007296 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:33.007301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:33.007304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:33.007333 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:33.010004 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.010673 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.010677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.010679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.010802 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.011220 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.011223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.011225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.011481 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:33.011586 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.012166 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.012315 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":635,"threads_started":1}
I20260926 07:41:33.013563 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:32.996403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:42587 (local address 127.11.9.67:33907)
0926 07:41:32.997751 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:32.997807 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:32.999584 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.000193 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.000202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.000625 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:41:33.001294 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.001317 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.002281 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.002285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.003470 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.003481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.004166 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.004170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.004355 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.005043 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:41:33.005064 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:41:33.007125 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:41:33.010125 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:33.010135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:33.010152 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:33.010533 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:33.010907 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:41:33.010915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:41:33.010919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:41:33.011118 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:41:33.011582 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:41:33.011588 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:41:33.011869 (+ 281us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.012078 (+ 209us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.012422 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":878,"threads_started":1}
I20260926 07:41:33.014868 23111 heartbeater.cc:347] Connected to a master server at 127.11.9.126:42587
I20260926 07:41:33.015164 23111 heartbeater.cc:464] Registering TS with master...
I20260926 07:41:33.015854 23111 heartbeater.cc:511] Master 127.11.9.126:42587 requested a full tablet report, sending...
I20260926 07:41:33.017309 22621 ts_manager.cc:194] Registered new tserver with Master: 38ce1537de2e4e67833b1cb103b18f55 (127.11.9.67:44119)
I20260926 07:41:33.018155 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:33907
I20260926 07:41:33.023042 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:41:33.039556 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.027716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.11.9.126:42587)
0926 07:41:33.027865 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.027870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.028367 (+ 497us) server_negotiation.cc:407] Connection header received
0926 07:41:33.028542 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.028549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.028642 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.028721 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:33.030158 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.030845 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.032312 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.032703 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.032793 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.035062 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:33.035066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:33.035069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:33.035095 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:33.037447 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.038070 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.038074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.038076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.038190 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.038486 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.038489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.038491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.038724 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:33.038903 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.039133 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.039323 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 07:41:33.043411 22621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:41:33.050563 22621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:41:33.088370 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.062454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:44119 (local address 127.0.0.1:49146)
0926 07:41:33.080077 (+ 17623us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:33.080105 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:33.080253 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.081073 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.081085 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.081122 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:41:33.081613 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.081624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.083196 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.083204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.085521 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.085539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.087328 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.087334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.087523 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.087571 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.087695 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.087892 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17467,"mutex_wait_us":5,"spinlock_wait_cycles":1408,"thread_start_us":17703,"threads_started":1}
I20260926 07:41:33.088550 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.065089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.11.9.65:44543)
0926 07:41:33.070887 (+ 5798us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.070901 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.080605 (+ 9704us) server_negotiation.cc:407] Connection header received
0926 07:41:33.080797 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.080803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.081048 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.081282 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:41:33.082088 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.083736 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.086252 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.087451 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.087598 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.087822 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.088051 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.088222 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5564,"thread_start_us":5397,"threads_started":1}
I20260926 07:41:33.088649 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.064387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44543 (local address 127.0.0.1:49860)
0926 07:41:33.080472 (+ 16085us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:33.080495 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:33.080618 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.081265 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.081271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.081301 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:41:33.081851 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.081861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.083878 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.083888 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.086091 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.086103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.087655 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.087662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.087838 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.087877 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.087995 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.088116 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15944,"mutex_wait_us":3,"thread_start_us":15387,"threads_started":1}
I20260926 07:41:33.089635 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.062123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:43175 (local address 127.0.0.1:48244)
0926 07:41:33.077747 (+ 15624us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:33.077816 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:33.078059 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.082131 (+ 4072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.082135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.082164 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:41:33.082513 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.082524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.084244 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.084250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.086781 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.086794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.088855 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.088861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.089035 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.089075 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.089185 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.089320 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15269,"mutex_wait_us":55,"thread_start_us":16195,"threads_started":1}
I20260926 07:41:33.089819 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.062913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.11.9.67:44119)
0926 07:41:33.079700 (+ 16787us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.079713 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.080404 (+ 691us) server_negotiation.cc:407] Connection header received
0926 07:41:33.080540 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.080549 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.080826 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.081038 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:41:33.081820 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.083067 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.085709 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.086819 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.086969 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.087206 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.089307 (+ 2101us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.089471 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16547,"thread_start_us":16811,"threads_started":1}
I20260926 07:41:33.090524 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.062346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.11.9.66:43175)
0926 07:41:33.080773 (+ 18427us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.080781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.080806 (+ 25us) server_negotiation.cc:407] Connection header received
0926 07:41:33.080909 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.080918 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.081936 (+ 1018us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.082120 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:41:33.082716 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.084105 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.086992 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.088400 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.088545 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.088705 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.090164 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.090308 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18269,"thread_start_us":18248,"threads_started":1}
I20260926 07:41:33.093039 23046 tablet_service.cc:1467] Processing CreateTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed (DEFAULT_TABLE table=test-table [id=b26c9f963c60484292479d81675ccc41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:41:33.093086 22769 tablet_service.cc:1467] Processing CreateTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed (DEFAULT_TABLE table=test-table [id=b26c9f963c60484292479d81675ccc41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:41:33.093811 22912 tablet_service.cc:1467] Processing CreateTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed (DEFAULT_TABLE table=test-table [id=b26c9f963c60484292479d81675ccc41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:41:33.094550 23046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985e78336e2f4e8dad1fa57ca73781ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:41:33.094904 22769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985e78336e2f4e8dad1fa57ca73781ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:41:33.094969 22912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985e78336e2f4e8dad1fa57ca73781ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:41:33.111877 23130 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Bootstrap starting.
I20260926 07:41:33.116222 23131 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Bootstrap starting.
I20260926 07:41:33.118521 23130 tablet_bootstrap.cc:654] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Neither blocks nor log segments found. Creating new log.
I20260926 07:41:33.120165 23130 log.cc:824] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Log is configured to *not* fsync() on all Append() calls
I20260926 07:41:33.120543 23131 tablet_bootstrap.cc:654] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Neither blocks nor log segments found. Creating new log.
I20260926 07:41:33.121299 23132 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Bootstrap starting.
I20260926 07:41:33.122525 23131 log.cc:824] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Log is configured to *not* fsync() on all Append() calls
I20260926 07:41:33.126431 23132 tablet_bootstrap.cc:654] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Neither blocks nor log segments found. Creating new log.
I20260926 07:41:33.127588 23132 log.cc:824] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Log is configured to *not* fsync() on all Append() calls
I20260926 07:41:33.128155 23130 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: No bootstrap required, opened a new log
I20260926 07:41:33.128522 23130 ts_tablet_manager.cc:1397] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260926 07:41:33.130693 23132 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: No bootstrap required, opened a new log
I20260926 07:41:33.130949 23131 tablet_bootstrap.cc:492] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: No bootstrap required, opened a new log
I20260926 07:41:33.130947 23132 ts_tablet_manager.cc:1397] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 07:41:33.131209 23131 ts_tablet_manager.cc:1397] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260926 07:41:33.135748 23130 raft_consensus.cc:348] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.136113 23130 raft_consensus.cc:374] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:41:33.136226 23130 raft_consensus.cc:729] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ce1537de2e4e67833b1cb103b18f55, State: Initialized, Role: FOLLOWER
I20260926 07:41:33.136831 23130 consensus_queue.cc:260] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.137957 23111 heartbeater.cc:503] Master 127.11.9.126:42587 was elected leader, sending a full tablet report...
I20260926 07:41:33.138652 23130 ts_tablet_manager.cc:1428] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260926 07:41:33.141635 23131 raft_consensus.cc:348] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.142127 23131 raft_consensus.cc:374] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:41:33.142277 23131 raft_consensus.cc:729] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caafd54a97b7463fb1406793207c9a5b, State: Initialized, Role: FOLLOWER
I20260926 07:41:33.141635 23132 raft_consensus.cc:348] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.142572 23132 raft_consensus.cc:374] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:41:33.142683 23132 raft_consensus.cc:729] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 165d4fbca3e34f6cab5449c12e022310, State: Initialized, Role: FOLLOWER
I20260926 07:41:33.143033 23131 consensus_queue.cc:260] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.143198 23132 consensus_queue.cc:260] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.145046 23131 ts_tablet_manager.cc:1428] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260926 07:41:33.145285 22977 heartbeater.cc:503] Master 127.11.9.126:42587 was elected leader, sending a full tablet report...
I20260926 07:41:33.145992 23132 ts_tablet_manager.cc:1428] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260926 07:41:33.184767 22835 tablet.cc:2369] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:41:33.187321 22978 tablet.cc:2369] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:41:33.193032 23136 raft_consensus.cc:482] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:41:33.193361 23136 raft_consensus.cc:504] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.194921 23136 leader_election.cc:288] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 165d4fbca3e34f6cab5449c12e022310 (127.11.9.66:43175), caafd54a97b7463fb1406793207c9a5b (127.11.9.65:44543)
I20260926 07:41:33.200363 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.195359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:60241 (local address 127.11.9.65:44543)
0926 07:41:33.195553 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.195561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.195580 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:41:33.195793 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.195797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.195855 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.195960 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:41:33.196348 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.197074 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.198707 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.199493 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.199638 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.199736 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.199983 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.200106 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:41:33.200362 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.195230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44543 (local address 127.11.9.67:60241)
0926 07:41:33.195464 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:33.195486 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:33.195636 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.195978 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.195982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.196007 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:41:33.196202 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.196214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.197252 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.197257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.198550 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.198565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.199645 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.199651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.199826 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.199849 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.199956 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.200083 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 07:41:33.201550 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.195317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:43619 (local address 127.11.9.66:43175)
0926 07:41:33.195506 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.195510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.196530 (+ 1020us) server_negotiation.cc:407] Connection header received
0926 07:41:33.196709 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.196721 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.196788 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.196894 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:41:33.197492 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.198162 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.199735 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.200582 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.200697 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.200794 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.201268 (+ 474us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.201363 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:41:33.201683 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.195719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:43175 (local address 127.11.9.67:43619)
0926 07:41:33.196425 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0926 07:41:33.196438 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 07:41:33.196580 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:41:33.196906 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:41:33.196910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:41:33.196931 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:41:33.197345 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.197355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.198313 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.198318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.199587 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:41:33.199598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.200887 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.200893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:41:33.201067 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.201099 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:41:33.201223 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 07:41:33.201352 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":386,"threads_started":1}
I20260926 07:41:33.201462 22789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985e78336e2f4e8dad1fa57ca73781ed" candidate_uuid: "38ce1537de2e4e67833b1cb103b18f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caafd54a97b7463fb1406793207c9a5b" is_pre_election: true
I20260926 07:41:33.202044 22789 raft_consensus.cc:2454] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ce1537de2e4e67833b1cb103b18f55 in term 0.
I20260926 07:41:33.202728 22999 leader_election.cc:302] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ce1537de2e4e67833b1cb103b18f55, caafd54a97b7463fb1406793207c9a5b; no voters:
I20260926 07:41:33.202689 22932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985e78336e2f4e8dad1fa57ca73781ed" candidate_uuid: "38ce1537de2e4e67833b1cb103b18f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165d4fbca3e34f6cab5449c12e022310" is_pre_election: true
I20260926 07:41:33.203090 23136 raft_consensus.cc:2781] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:41:33.203218 23136 raft_consensus.cc:482] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:41:33.203210 22932 raft_consensus.cc:2454] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ce1537de2e4e67833b1cb103b18f55 in term 0.
I20260926 07:41:33.203312 23136 raft_consensus.cc:3037] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:41:33.205578 23136 raft_consensus.cc:504] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.206312 23136 leader_election.cc:288] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [CANDIDATE]: Term 1 election: Requested vote from peers 165d4fbca3e34f6cab5449c12e022310 (127.11.9.66:43175), caafd54a97b7463fb1406793207c9a5b (127.11.9.65:44543)
I20260926 07:41:33.206656 22932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985e78336e2f4e8dad1fa57ca73781ed" candidate_uuid: "38ce1537de2e4e67833b1cb103b18f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "165d4fbca3e34f6cab5449c12e022310"
I20260926 07:41:33.206743 22789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985e78336e2f4e8dad1fa57ca73781ed" candidate_uuid: "38ce1537de2e4e67833b1cb103b18f55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caafd54a97b7463fb1406793207c9a5b"
I20260926 07:41:33.206869 22932 raft_consensus.cc:3037] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:41:33.206940 22789 raft_consensus.cc:3037] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:41:33.210098 22789 raft_consensus.cc:2454] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ce1537de2e4e67833b1cb103b18f55 in term 1.
I20260926 07:41:33.210101 22932 raft_consensus.cc:2454] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ce1537de2e4e67833b1cb103b18f55 in term 1.
I20260926 07:41:33.210500 23000 leader_election.cc:302] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 165d4fbca3e34f6cab5449c12e022310, 38ce1537de2e4e67833b1cb103b18f55; no voters:
I20260926 07:41:33.210820 23136 raft_consensus.cc:2781] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:41:33.211282 23136 raft_consensus.cc:686] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 1 LEADER]: Becoming Leader. State: Replica: 38ce1537de2e4e67833b1cb103b18f55, State: Running, Role: LEADER
I20260926 07:41:33.211750 23136 consensus_queue.cc:237] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } }
I20260926 07:41:33.216190 22620 catalog_manager.cc:5495] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ce1537de2e4e67833b1cb103b18f55 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "38ce1537de2e4e67833b1cb103b18f55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ce1537de2e4e67833b1cb103b18f55" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 44119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 } health_report { overall_health: UNKNOWN } } }
W20260926 07:41:33.244869 23112 tablet.cc:2369] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:41:33.264955 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.259684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.11.9.67:44119)
0926 07:41:33.259875 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.259883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.259903 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:41:33.260001 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.260004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.260074 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.260228 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:41:33.260618 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.261391 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.263008 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.263382 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.263482 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.263910 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:41:33.264410 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:41:33.264520 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.264613 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.264673 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:41:33.272832 22932 raft_consensus.cc:1261] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 1 FOLLOWER]: Refusing update from remote peer 38ce1537de2e4e67833b1cb103b18f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:41:33.272831 22789 raft_consensus.cc:1261] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 1 FOLLOWER]: Refusing update from remote peer 38ce1537de2e4e67833b1cb103b18f55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:41:33.273870 23140 consensus_queue.cc:1035] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "165d4fbca3e34f6cab5449c12e022310" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:41:33.274224 23136 consensus_queue.cc:1035] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caafd54a97b7463fb1406793207c9a5b" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:41:33.287781 23144 mvcc.cc:205] Tried to move back new op lower bound from 7333513188435832832 to 7333513188199288832. Current Snapshot: MvccSnapshot[applied={T|T < 7333513188435832832 or (T in {7333513188435832832})}]
I20260926 07:41:33.290251 23145 mvcc.cc:205] Tried to move back new op lower bound from 7333513188435832832 to 7333513188199288832. Current Snapshot: MvccSnapshot[applied={T|T < 7333513188435832832}]
I20260926 07:41:33.297629 22621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:41:33.297873 22621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:41:33.301265 22621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93a53127e5384aceb3feb00022cb4561: Sending DeleteTablet for 3 replicas of tablet 985e78336e2f4e8dad1fa57ca73781ed
I20260926 07:41:33.302340 23046 tablet_service.cc:1514] Processing DeleteTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:41:33 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:49146
I20260926 07:41:33.302330 22912 tablet_service.cc:1514] Processing DeleteTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:41:33 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:48244
I20260926 07:41:33.302949 22769 tablet_service.cc:1514] Processing DeleteTablet for tablet 985e78336e2f4e8dad1fa57ca73781ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:41:33 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:49860
I20260926 07:41:33.305965 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 22701
I20260926 07:41:33.314673 23152 tablet_replica.cc:330] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: stopping tablet replica
I20260926 07:41:33.315187 23152 raft_consensus.cc:2229] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:41:33.315869 23152 raft_consensus.cc:2258] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:41:33.316594 23154 tablet_replica.cc:330] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: stopping tablet replica
I20260926 07:41:33.316972 23154 raft_consensus.cc:2229] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:41:33.317437 23154 raft_consensus.cc:2258] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:41:33.318325 23153 tablet_replica.cc:330] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: stopping tablet replica
I20260926 07:41:33.318674 23153 raft_consensus.cc:2229] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:41:33.319077 23153 raft_consensus.cc:2258] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:41:33.320469 23154 ts_tablet_manager.cc:1905] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:41:33.320647 23152 ts_tablet_manager.cc:1905] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:41:33.321049 23153 ts_tablet_manager.cc:1905] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:41:33.327507 23154 ts_tablet_manager.cc:1918] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:41:33.327708 23154 log.cc:1196] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/985e78336e2f4e8dad1fa57ca73781ed
I20260926 07:41:33.327838 23152 ts_tablet_manager.cc:1918] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:41:33.327917 23153 ts_tablet_manager.cc:1918] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:41:33.328022 23152 log.cc:1196] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/985e78336e2f4e8dad1fa57ca73781ed
I20260926 07:41:33.328267 23153 log.cc:1196] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/985e78336e2f4e8dad1fa57ca73781ed
I20260926 07:41:33.328316 23154 ts_tablet_manager.cc:1939] T 985e78336e2f4e8dad1fa57ca73781ed P caafd54a97b7463fb1406793207c9a5b: Deleting consensus metadata
I20260926 07:41:33.328550 23152 ts_tablet_manager.cc:1939] T 985e78336e2f4e8dad1fa57ca73781ed P 38ce1537de2e4e67833b1cb103b18f55: Deleting consensus metadata
I20260926 07:41:33.328888 23153 ts_tablet_manager.cc:1939] T 985e78336e2f4e8dad1fa57ca73781ed P 165d4fbca3e34f6cab5449c12e022310: Deleting consensus metadata
I20260926 07:41:33.330343 22608 catalog_manager.cc:4841] TS 38ce1537de2e4e67833b1cb103b18f55 (127.11.9.67:44119): tablet 985e78336e2f4e8dad1fa57ca73781ed (table test-table [id=b26c9f963c60484292479d81675ccc41]) successfully deleted
I20260926 07:41:33.330479 22605 catalog_manager.cc:4841] TS caafd54a97b7463fb1406793207c9a5b (127.11.9.65:44543): tablet 985e78336e2f4e8dad1fa57ca73781ed (table test-table [id=b26c9f963c60484292479d81675ccc41]) successfully deleted
I20260926 07:41:33.330545 22606 catalog_manager.cc:4841] TS 165d4fbca3e34f6cab5449c12e022310 (127.11.9.66:43175): tablet 985e78336e2f4e8dad1fa57ca73781ed (table test-table [id=b26c9f963c60484292479d81675ccc41]) successfully deleted
Sep 26 07:41:33 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:41:33.339234 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.307870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.11.9.65:44543)
0926 07:41:33.308101 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.308111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.318463 (+ 10352us) server_negotiation.cc:407] Connection header received
0926 07:41:33.318532 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.318537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.318649 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.318785 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:33.320691 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.321837 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.323871 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.324519 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.324624 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.332305 (+ 7681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:33.332319 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:33.332338 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:33.332384 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:33.336215 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.336923 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.336932 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.336935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.337087 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.337565 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.337570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.337574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.338017 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:33.338318 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.338571 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.338814 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":79}
I20260926 07:41:33.340060 22829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:41:33.429001 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 22701
I20260926 07:41:33.448619 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 22840
Sep 26 07:41:33 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:41:33.464473 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.449461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.11.9.66:43175)
0926 07:41:33.449622 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.449629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.449995 (+ 366us) server_negotiation.cc:407] Connection header received
0926 07:41:33.450189 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.450195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.450303 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.450410 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:33.451646 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.452408 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.453755 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.454153 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.454302 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.458311 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:33.458322 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:33.458335 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:33.458364 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:33.461640 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.462426 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.462437 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.462446 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.462600 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.462987 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.462994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.462998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.463352 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:33.463637 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.463924 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.464128 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 07:41:33.465368 22972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:41:33.555570 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 22840
I20260926 07:41:33.572984 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 22983
Sep 26 07:41:33 dist-test-slave-x492 krb5kdc[22145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:41:33.589344 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:41:33.573794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.11.9.67:44119)
0926 07:41:33.573969 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 07:41:33.573973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:41:33.574199 (+ 226us) server_negotiation.cc:407] Connection header received
0926 07:41:33.574394 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:41:33.574399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:41:33.574485 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:41:33.574615 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:41:33.575918 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.576872 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.578369 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:41:33.578839 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:41:33.578982 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:41:33.583115 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:41:33.583127 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:41:33.583139 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:41:33.583180 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:41:33.586350 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.587064 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.587072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.587075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.587234 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:41:33.587624 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:41:33.587631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:41:33.587634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:41:33.588062 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:41:33.588371 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 07:41:33.588702 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 07:41:33.588915 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:41:33.590013 23106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:41:33.709218 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 22983
I20260926 07:41:33.728081 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 22589
I20260926 07:41:33.729504 22651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:41:33.842460 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 22589
I20260926 07:41:33.868110 11301 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 07:41:34.203728 11301 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 07:41:34.203963 11301 mini_ranger.cc:66] Stopping Ranger...
I20260926 07:41:34.548774 11301 mini_ranger.cc:68] Stopped Ranger
2026-09-26 07:41:34.549 UTC [22175] LOG: received smart shutdown request
2026-09-26 07:41:34.552 UTC [22175] LOG: background worker "logical replication launcher" (PID 22184) exited with exit code 1
2026-09-26 07:41:34.552 UTC [22179] LOG: shutting down
2026-09-26 07:41:34.737 UTC [22175] LOG: database system is shut down
2026-09-26T07:41:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48411 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:41:34 dist-test-slave-x492 krb5kdc[23168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:41:34 dist-test-slave-x492 krb5kdc[23168](info): set up 2 sockets
Sep 26 07:41:34 dist-test-slave-x492 krb5kdc[23168](info): commencing operation
krb5kdc: starting...
W20260926 07:41:35.957597 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:41:36 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:41:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangeradmin_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangeradmin_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangerlookup_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangerlookup_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
I20260926 07:41:36.278580 11301 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm0hQhn/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/postgres -l logfile start
2026-09-26 07:41:38.747 UTC [23198] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 07:41:38.747 UTC [23198] LOG: listening on IPv4 address "127.11.9.84", port 42355
2026-09-26 07:41:38.750 UTC [23198] LOG: listening on Unix socket "/tmp/.s.PGSQL.42355"
2026-09-26 07:41:38.773 UTC [23201] LOG: database system was shut down at 2026-09-26 07:41:37 UTC
2026-09-26 07:41:38.777 UTC [23198] LOG: database system is ready to accept connections
I20260926 07:41:39.753228 11301 mini_postgres.cc:96] Postgres bound to 42355
2026-09-26 07:41:39.774 UTC [23209] FATAL: database "slave" does not exist
127.11.9.84:42355 - accepting connections
I20260926 07:41:39.774713 11301 mini_ranger.cc:161] Starting Ranger...
I20260926 07:41:39.799247 11301 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 07:41:40.287716 11301 mini_ranger.cc:87] Created ranger Postgres database
I20260926 07:41:40.287859 11301 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin
2026-09-26 07:41:40,575 [I] DB FLAVOR :POSTGRES
2026-09-26 07:41:40,576 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 07:41:40,576 [I] Checking connection..
2026-09-26 07:41:40,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 07:41:40,824 [I] Checking connection passed.
2026-09-26 07:41:40,824 [I] --------- Verifying version history table ---------
2026-09-26 07:41:40,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:41:41,063 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 07:41:41,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:41:41,309 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 07:41:41,309 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 07:41:41,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 07:41:41.565 UTC [23273] WARNING: there is no transaction in progress
2026-09-26 07:41:41,572 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 07:41:41,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 07:41:41,793 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 07:41:41,794 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 07:41:41,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 07:41:42,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 07:41:42,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x492.c.gcp-upstream.internal','N') ;"
2026-09-26 07:41:42,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 07:41:42,748 [I] Table x_portal_user does not exist in database ranger
2026-09-26 07:41:42,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 07:41:42,980 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 07:41:42,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 07:41:43,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 07:41:43,447 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 07:41:43,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 07:41:44.373 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.389 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.405 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.420 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.439 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.458 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.522 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.529 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.543 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.553 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.563 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.576 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.586 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.592 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.599 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:44.606 UTC [23381] WARNING: there is no transaction in progress
2026-09-26 07:41:45,227 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 07:41:45,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 07:41:45,477 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 07:41:45,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 07:41:45,735 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 07:41:45,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 07:41:45,976 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 07:41:45,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 07:41:46,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 07:41:46,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x492.c.gcp-upstream.internal';"
2026-09-26 07:41:46,741 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 07:41:46,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 07:41:46,985 [I] DB_PATCHES have already been applied
I20260926 07:41:46.991180 11301 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm0hQhn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 07:41:46.991266 11301 mini_ranger.cc:193] Using host: 127.11.9.84
I20260926 07:41:47.005429 11301 mini_ranger.cc:227] Ranger admin URL: http://127.11.9.84:55261
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 7:41:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 07:41:48.138931 11301 mini_ranger.cc:160] Time spent starting Ranger: real 8.364s user 0.001s sys 0.036s
Sep 26, 2026 7:42:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 7:42:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 26, 2026 7:42:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 7:42:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 7:42:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 07:42:09.854764 11301 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.9.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.9.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangerkms_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.9.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/rangerkms_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.9.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.9.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.9.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/HTTP_127.11.9.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 07:42:10 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 07:42:10.018122 11301 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 07:42:10.038633 11301 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 07:42:10.901684 11301 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 07:42:10.901782 11301 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms
I20260926 07:42:10.901810 11301 mini_ranger_kms.cc:227] Using postgres at 127.11.9.84:42355
2026-09-26 07:42:11,251 [I] DB FLAVOR :POSTGRES
2026-09-26 07:42:11,251 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 07:42:11,251 [I] Checking connection
2026-09-26 07:42:11,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 07:42:11,480 [I] connection success
2026-09-26 07:42:11,480 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 07:42:11,480 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 07:42:11,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 07:42:11,703 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 07:42:11,704 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 07:42:11,704 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 07:42:11,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.9.84:42355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 07:42:11,963 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 07:42:12.444605 11301 mini_ranger_kms.cc:310] Created kms service
I20260926 07:42:12.632697 11301 mini_ranger_kms.cc:326] Created kudu user
I20260926 07:42:12.679441 11301 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 07:42:12 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:13.052011 11301 mini_ranger_kms.cc:384] Added ranger policy
I20260926 07:42:13.052362 11301 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm0hQhn/build/asan/bin/postgresql.jar:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm0hQhn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 07:42:13.052450 11301 mini_ranger_kms.cc:240] Using host: 127.11.9.84
I20260926 07:42:13.070124 11301 mini_ranger_kms.cc:276] Ranger KMS PID: 23561
I20260926 07:42:13.070257 11301 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.9.84:48863
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 7:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 07:42:15 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790408535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:15.421900 11301 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.404s user 0.048s sys 0.314s
I20260926 07:42:15.422184 11301 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 07:42:15.422360 11301 mini_ranger_kms.cc:401] 127.11.9.84:48863/kms/v1/keys
Sep 26 07:42:15 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:42:16 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.9.84@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:42:18 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:42:18.372263 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45311
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45311
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:18.707209 23612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:18.707491 23612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:18.707634 23612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:18.715363 23612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:18.715466 23612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:18.715553 23612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:18.715647 23612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:18.715737 23612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:18.725838 23612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.9.84:48863/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45311
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:18.727137 23612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:18.728574 23612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:18.742072 23618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:18.742110 23620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:18.742606 23617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:18.743191 23612 server_base.cc:1034] running on GCE node
I20260926 07:42:18.744114 23612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:18.745894 23612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:18.747146 23612 hybrid_clock.cc:648] HybridClock initialized: now 1790408538747102 us; error 47 us; skew 500 ppm
I20260926 07:42:18.747556 23612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:18 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:18.751240 23612 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:18.752751 23612 webserver.cc:460] Webserver started at http://127.11.9.126:41799/ using document root <none> and password file <none>
I20260926 07:42:18.753461 23612 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:18.753583 23612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:18.753887 23612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:42:18 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:18.810575 23612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "68c688258aa94e049561d73d2250a2a1"
format_stamp: "Formatted at 2026-09-26 07:42:18 on dist-test-slave-x492"
server_key: "776a99f42cd7228a16869993f61602ad"
server_key_iv: "69239ee28caffbe24ae6fb2990c831d2"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:18.811332 23612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "68c688258aa94e049561d73d2250a2a1"
format_stamp: "Formatted at 2026-09-26 07:42:18 on dist-test-slave-x492"
server_key: "776a99f42cd7228a16869993f61602ad"
server_key_iv: "69239ee28caffbe24ae6fb2990c831d2"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:18.816779 23612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:42:18.837284 23627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:18.839210 23612 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 07:42:18.839411 23612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "68c688258aa94e049561d73d2250a2a1"
format_stamp: "Formatted at 2026-09-26 07:42:18 on dist-test-slave-x492"
server_key: "776a99f42cd7228a16869993f61602ad"
server_key_iv: "69239ee28caffbe24ae6fb2990c831d2"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:18.839672 23612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:18.869067 23612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:18.872591 23612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:18.872851 23612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:18.891396 23612 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45311
I20260926 07:42:18.891434 23678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45311 every 8 connection(s)
I20260926 07:42:18.892719 23612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:18.894616 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 23612
I20260926 07:42:18.894834 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:42:18.895524 11301 external_mini_cluster.cc:1428] Setting key 5d40b3de06fd08a03cacb3b9dc3c2887
I20260926 07:42:18.896826 23679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 07:42:18 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:18.910676 23679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Bootstrap starting.
I20260926 07:42:18.913846 23679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:18.914842 23679 log.cc:824] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:18.917827 23679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: No bootstrap required, opened a new log
I20260926 07:42:18.918022 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:18.897470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.11.9.126:45311)
0926 07:42:18.898246 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:18.898257 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:18.898295 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:42:18.899822 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:18.899842 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:18.900228 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:18.900584 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:18.902330 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:18.903685 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:18.905491 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:18.906103 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:18.906231 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:18.910676 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:18.910683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:18.910697 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:18.910729 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:18.913756 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:18.914524 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:18.914531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:18.914536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:18.914685 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:18.915094 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:18.915100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:18.915103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:18.915832 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:18.916167 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:18.916596 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 07:42:18.917006 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":381,"threads_started":1}
I20260926 07:42:18.925879 23679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:18.926322 23679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:18.926426 23679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c688258aa94e049561d73d2250a2a1, State: Initialized, Role: FOLLOWER
I20260926 07:42:18.926900 23679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:18.927086 23679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:18.927177 23679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:18.927289 23679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:18.929309 23679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:18.929718 23679 leader_election.cc:302] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c688258aa94e049561d73d2250a2a1; no voters:
I20260926 07:42:18.930226 23679 leader_election.cc:288] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:42:18.930365 23684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:18.931098 23684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 LEADER]: Becoming Leader. State: Replica: 68c688258aa94e049561d73d2250a2a1, State: Running, Role: LEADER
I20260926 07:42:18.931684 23684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:18.932276 23679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:18.934700 23686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c688258aa94e049561d73d2250a2a1. Latest consensus state: current_term: 1 leader_uuid: "68c688258aa94e049561d73d2250a2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } } }
I20260926 07:42:18.935086 23686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:18.934738 23685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68c688258aa94e049561d73d2250a2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } } }
I20260926 07:42:18.935653 23685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:18.937903 23691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:18.944463 23691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:18.948628 23679 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 07:42:18.962340 23691 catalog_manager.cc:1347] Generated new cluster ID: e6b76197f6d34d108eb12dbf85777807
I20260926 07:42:18.962494 23691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:18.977833 23691 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:42:18.979204 23691 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:18.987242 23691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Generated new TSK 0
I20260926 07:42:18.988046 23691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:42:19.769057 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 07:42:20.174645 23724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:20.175011 23724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:20.175212 23724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:20.186801 23724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:20.186971 23724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:20.187177 23724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:42:20.202755 23724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:20.204470 23724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:20.206295 23724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:20.221531 23735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:20.221997 23734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:20.222050 23737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:20.223616 23724 server_base.cc:1034] running on GCE node
I20260926 07:42:20.224256 23724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:20.229473 23724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:20.230767 23724 hybrid_clock.cc:648] HybridClock initialized: now 1790408540230686 us; error 79 us; skew 500 ppm
I20260926 07:42:20.231109 23724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:20.234570 23724 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:42:20.241194 23724 webserver.cc:460] Webserver started at http://127.11.9.65:44959/ using document root <none> and password file <none>
I20260926 07:42:20.241933 23724 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:20.242074 23724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:20.242416 23724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:20.260510 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "531b8db911e54c57a5616ecfd376b679"
format_stamp: "Formatted at 2026-09-26 07:42:20 on dist-test-slave-x492"
server_key: "74bf5e6a0e13435164e61398b7d5b55f"
server_key_iv: "befb2edff8e6a04875194388a17ee03e"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:20.262010 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "531b8db911e54c57a5616ecfd376b679"
format_stamp: "Formatted at 2026-09-26 07:42:20 on dist-test-slave-x492"
server_key: "74bf5e6a0e13435164e61398b7d5b55f"
server_key_iv: "befb2edff8e6a04875194388a17ee03e"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:20.271719 23724 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260926 07:42:20.307279 23744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:20.310705 23724 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.009s sys 0.004s
I20260926 07:42:20.311004 23724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "531b8db911e54c57a5616ecfd376b679"
format_stamp: "Formatted at 2026-09-26 07:42:20 on dist-test-slave-x492"
server_key: "74bf5e6a0e13435164e61398b7d5b55f"
server_key_iv: "befb2edff8e6a04875194388a17ee03e"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:20.311362 23724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:20.373529 23724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:20.378825 23724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:20.379163 23724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:20.381585 23724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:20.390059 23724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:20.390161 23724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:20.390282 23724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:20.390343 23724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:20.487267 23724 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:35479
I20260926 07:42:20.487296 23856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:35479 every 8 connection(s)
I20260926 07:42:20.489147 23724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:42:20.497438 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 23724
I20260926 07:42:20.497669 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:42:20.498582 11301 external_mini_cluster.cc:1428] Setting key 5e9574402439697b4ecc39b29dff9f75
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:42:20.551285 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:20.500479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.65:45003)
0926 07:42:20.515535 (+ 15056us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:20.515606 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:20.518324 (+ 2718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:20.526599 (+ 8275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:20.526617 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:20.527385 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:20.528377 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:20.528411 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:20.531987 (+ 3576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:20.531997 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:20.534806 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:20.534827 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:20.535974 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:20.535985 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:20.536354 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:20.537286 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:20.537328 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:20.540446 (+ 3118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:20.546520 (+ 6074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:20.546530 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:20.546552 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:20.547145 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:20.547588 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:20.547593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:20.547597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:20.547833 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:20.548564 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:20.548573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:20.548996 (+ 423us) client_negotiation.cc:769] Sending connection context
0926 07:42:20.549315 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 07:42:20.549884 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13449,"thread_start_us":13161,"threads_started":1}
I20260926 07:42:20.552388 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:20.501147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:45003 (local address 127.11.9.126:45311)
0926 07:42:20.513639 (+ 12492us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:20.513646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:20.515782 (+ 2136us) server_negotiation.cc:407] Connection header received
0926 07:42:20.520346 (+ 4564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:20.520352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:20.520462 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:20.520591 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:20.528609 (+ 8018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:20.529848 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:20.535012 (+ 5164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:20.535668 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:20.535797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:20.541080 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:20.541087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:20.541092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:20.541150 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:20.544930 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:20.547298 (+ 2368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:20.547303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:20.547307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:20.547458 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:20.547977 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:20.547982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:20.547986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:20.548348 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:20.548487 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:20.551793 (+ 3306us) server_negotiation.cc:299] Negotiation successful
0926 07:42:20.552027 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12346,"thread_start_us":12208,"threads_started":1}
I20260926 07:42:20.555567 23857 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:20.555939 23857 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:20.556895 23857 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:20.559777 23644 ts_manager.cc:194] Registered new tserver with Master: 531b8db911e54c57a5616ecfd376b679 (127.11.9.65:35479)
I20260926 07:42:20.562218 23644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:45003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:42:20.589722 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
Sep 26 07:42:20 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408540, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
W20260926 07:42:21.015084 23863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:21.015611 23863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:21.015811 23863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:21.027482 23863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:21.027684 23863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:21.027892 23863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:42:21.043897 23863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:21.045576 23863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:21.047370 23863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:21.071223 23870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:21.072929 23871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:21.076462 23874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:21.077309 23863 server_base.cc:1034] running on GCE node
I20260926 07:42:21.078153 23863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:21.085290 23863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:21.086623 23863 hybrid_clock.cc:648] HybridClock initialized: now 1790408541086559 us; error 66 us; skew 500 ppm
I20260926 07:42:21.087083 23863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:21.091611 23863 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:42:21.097283 23863 webserver.cc:460] Webserver started at http://127.11.9.66:41369/ using document root <none> and password file <none>
I20260926 07:42:21.098227 23863 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:21.098438 23863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:21.098820 23863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:21.121048 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "fc39b69cbc6445f7a019515cd34c82ac"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "d785daab515796e5235fce09e2bc6ec7"
server_key_iv: "930298ef98b2bbd4e740a782e08a38be"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.122396 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "fc39b69cbc6445f7a019515cd34c82ac"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "d785daab515796e5235fce09e2bc6ec7"
server_key_iv: "930298ef98b2bbd4e740a782e08a38be"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.134406 23863 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.001s
I20260926 07:42:21.169896 23880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:21.172027 23863 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.012s sys 0.004s
I20260926 07:42:21.172232 23863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "fc39b69cbc6445f7a019515cd34c82ac"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "d785daab515796e5235fce09e2bc6ec7"
server_key_iv: "930298ef98b2bbd4e740a782e08a38be"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.172501 23863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:21.203293 23863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:21.207800 23863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:21.208187 23863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:21.215548 23863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:21.224054 23863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:21.224215 23863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 07:42:21.224386 23863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:21.224485 23863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:21.278420 23863 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:38167
I20260926 07:42:21.278522 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:38167 every 8 connection(s)
I20260926 07:42:21.279838 23863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:42:21.289033 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 23863
I20260926 07:42:21.289256 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:42:21.289925 11301 external_mini_cluster.cc:1428] Setting key fdaff0817b7dbccf0975e423c89644ed
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
I20260926 07:42:21.330466 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.284487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40811 (local address 127.11.9.126:45311)
0926 07:42:21.293635 (+ 9148us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:21.293642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:21.306739 (+ 13097us) server_negotiation.cc:407] Connection header received
0926 07:42:21.309492 (+ 2753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:21.309497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:21.309614 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:21.309720 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:21.311924 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.313137 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.315831 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.316579 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.316685 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:21.321842 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:21.321851 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:21.321857 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:21.321918 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:21.325525 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.326348 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.326353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.326358 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.326477 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.326908 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.326913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.326916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.327210 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:21.327370 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:21.329686 (+ 2316us) server_negotiation.cc:299] Negotiation successful
0926 07:42:21.329949 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9003,"thread_start_us":8887,"threads_started":1}
I20260926 07:42:21.332934 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.283785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.66:40811)
0926 07:42:21.306209 (+ 22424us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:21.306264 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:21.309145 (+ 2881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:21.310158 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:21.310173 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:21.310861 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:21.311711 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:21.311739 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.313365 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.313373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:21.315663 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:21.315685 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.317050 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.317060 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:21.317427 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:21.318308 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:21.318343 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:21.321564 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:21.325670 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:21.325678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:21.325698 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:21.326210 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:21.326586 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:21.326592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:21.326595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:21.326786 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:21.327356 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:21.327363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:21.327856 (+ 493us) client_negotiation.cc:769] Sending connection context
0926 07:42:21.330805 (+ 2949us) client_negotiation.cc:240] Negotiation successful
0926 07:42:21.331411 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22094,"thread_start_us":21816,"threads_started":1}
I20260926 07:42:21.334636 24000 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:21.334975 24000 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:21.335680 24000 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:21.337106 23644 ts_manager.cc:194] Registered new tserver with Master: fc39b69cbc6445f7a019515cd34c82ac (127.11.9.66:38167)
I20260926 07:42:21.337931 23644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:40811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:42:21.360430 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:21.566280 23857 heartbeater.cc:503] Master 127.11.9.126:45311 was elected leader, sending a full tablet report...
W20260926 07:42:21.692646 24006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:21.692960 24006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:21.693132 24006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:21.700409 24006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:21.700541 24006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:21.700714 24006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:42:21.710340 24006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:21.711532 24006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:21.712776 24006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:21.726063 24011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:21.726016 24012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:21.726102 24014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:21.727326 24006 server_base.cc:1034] running on GCE node
I20260926 07:42:21.727913 24006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:21.728857 24006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:21.730084 24006 hybrid_clock.cc:648] HybridClock initialized: now 1790408541730036 us; error 57 us; skew 500 ppm
I20260926 07:42:21.730432 24006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:21.740301 24006 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:42:21.741698 24006 webserver.cc:460] Webserver started at http://127.11.9.67:42529/ using document root <none> and password file <none>
I20260926 07:42:21.742331 24006 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:21.742429 24006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:21.742669 24006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:21.756057 24006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "bc484a8b1e9e4517b23c79420ce61463"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "2e228b29800d3cf8bd7425dfe5ed43db"
server_key_iv: "0fe354efb0cb18267b021be0290ca4b0"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.756855 24006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "bc484a8b1e9e4517b23c79420ce61463"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "2e228b29800d3cf8bd7425dfe5ed43db"
server_key_iv: "0fe354efb0cb18267b021be0290ca4b0"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.762223 24006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 07:42:21.775969 24021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:21.777700 24006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:42:21.777892 24006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "bc484a8b1e9e4517b23c79420ce61463"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "2e228b29800d3cf8bd7425dfe5ed43db"
server_key_iv: "0fe354efb0cb18267b021be0290ca4b0"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:21.778136 24006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:21.792166 24006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:21.796095 24006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:21.796401 24006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:21.797365 24006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:21.798627 24006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:21.798728 24006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:21.798872 24006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:21.798955 24006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:21.830556 24006 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:33535
I20260926 07:42:21.830596 24133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:33535 every 8 connection(s)
I20260926 07:42:21.831852 24006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:42:21.835965 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24006
I20260926 07:42:21.836139 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:42:21.836736 11301 external_mini_cluster.cc:1428] Setting key 0408a103aa2716d2975e0ff5cfc769f1
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408541, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:21.851198 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.834442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:60783 (local address 127.11.9.126:45311)
0926 07:42:21.834900 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:21.834904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:21.835844 (+ 940us) server_negotiation.cc:407] Connection header received
0926 07:42:21.838003 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:21.838008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:21.838089 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:21.838184 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:21.839681 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.840397 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.841967 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.842384 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.842516 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:21.845805 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:21.845810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:21.845814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:21.845843 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:21.848330 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.848980 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.848984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.848986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.849097 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.849542 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.849545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.849547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.849795 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:21.849903 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:21.850598 (+ 695us) server_negotiation.cc:299] Negotiation successful
0926 07:42:21.850827 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":210,"threads_started":1}
I20260926 07:42:21.852350 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.834945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.67:60783)
0926 07:42:21.835635 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:21.835695 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:21.837784 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:21.838410 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:21.838420 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:21.838956 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:21.839494 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:21.839511 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.840540 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.840546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:21.841816 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:21.841828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.842595 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.842601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:21.842828 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:21.843426 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:21.843446 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:21.845577 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:21.848446 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:21.848452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:21.848470 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:21.848877 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:21.849245 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:21.849249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:21.849251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:21.849443 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:21.849908 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:21.849913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:21.850169 (+ 256us) client_negotiation.cc:769] Sending connection context
0926 07:42:21.850491 (+ 322us) client_negotiation.cc:240] Negotiation successful
0926 07:42:21.850983 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":269,"threads_started":1}
I20260926 07:42:21.853615 24134 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:21.853907 24134 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:21.854549 24134 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:21.855825 23644 ts_manager.cc:194] Registered new tserver with Master: bc484a8b1e9e4517b23c79420ce61463 (127.11.9.67:33535)
I20260926 07:42:21.856632 23644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:60783
I20260926 07:42:21.864125 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:42:21.865823 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 23724
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:42:21.882189 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.866771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.11.9.65:35479)
0926 07:42:21.867389 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:21.867395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:21.867417 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:42:21.867473 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:21.867479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:21.867657 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:21.867785 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:21.869232 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.870232 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.871715 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.872224 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.872323 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:21.876395 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:21.876405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:21.876417 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:21.876450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:21.879570 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.880339 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.880348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.880351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.880515 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:21.880821 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:21.880824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:21.880826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:21.881144 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:21.881465 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:21.881720 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 07:42:21.881921 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":312,"threads_started":1}
I20260926 07:42:21.883576 23852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:21.971704 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 23724
I20260926 07:42:21.990518 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 23863
Sep 26 07:42:21 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:42:22.005719 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:21.991352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.11.9.66:38167)
0926 07:42:21.991908 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:21.991915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:21.991941 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:42:21.992001 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:21.992007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:21.992233 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:21.992381 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:21.993546 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.994516 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.995725 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:21.996291 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:21.996386 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:22.000530 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:22.000541 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:22.000555 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:22.000593 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:22.003335 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:22.003971 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:22.003978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:22.003980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:22.004108 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:22.004406 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:22.004409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:22.004411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:22.004728 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:22.004949 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:22.005212 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 07:42:22.005438 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":291,"threads_started":1}
I20260926 07:42:22.006956 23995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:22.088732 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 23863
I20260926 07:42:22.104215 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24006
Sep 26 07:42:22 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:42:22.120671 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:22.105244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.11.9.67:33535)
0926 07:42:22.105849 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:22.105857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:22.105884 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:42:22.105942 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:22.105949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:22.106162 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:22.106351 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:22.107619 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:22.108647 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:22.110155 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:22.110710 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:22.110855 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:22.115074 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:22.115084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:22.115098 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:22.115131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:22.117919 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:22.118623 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:22.118629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:22.118631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:22.118754 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:22.119119 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:22.119128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:22.119130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:22.119446 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:22.119754 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:22.120055 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 07:42:22.120319 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
I20260926 07:42:22.121975 24129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:22.203970 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24006
I20260926 07:42:22.219681 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 23612
I20260926 07:42:22.220924 23674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:22.326193 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 23612
I20260926 07:42:22.350093 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45311
--webserver_interface=127.11.9.126
--webserver_port=41799
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45311
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:22.670276 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:22.670552 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:22.670727 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:22.677881 24146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:22.678009 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:22.678115 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:22.678221 24146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:22.678313 24146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:22.688213 24146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45311
--ranger_config_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=41799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:22.689494 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:22.690810 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:22.702750 24151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:22.702791 24154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:22.702724 24152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:22.703435 24146 server_base.cc:1034] running on GCE node
I20260926 07:42:22.704231 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:22.705684 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:22.706919 24146 hybrid_clock.cc:648] HybridClock initialized: now 1790408542706884 us; error 42 us; skew 500 ppm
I20260926 07:42:22.707258 24146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:22 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408542, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:22.710410 24146 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:22.711959 24146 webserver.cc:460] Webserver started at http://127.11.9.126:41799/ using document root <none> and password file <none>
I20260926 07:42:22.712598 24146 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:22.712885 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 07:42:22 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408542, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:22.784005 24146 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:42:22.787096 24162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:22.788439 24146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:42:22.788646 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "68c688258aa94e049561d73d2250a2a1"
format_stamp: "Formatted at 2026-09-26 07:42:18 on dist-test-slave-x492"
server_key: "776a99f42cd7228a16869993f61602ad"
server_key_iv: "69239ee28caffbe24ae6fb2990c831d2"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:22.789664 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:22.806751 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:22.810132 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:22.810420 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:22.828184 24146 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45311
I20260926 07:42:22.828204 24213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45311 every 8 connection(s)
I20260926 07:42:22.829675 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:22.832156 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24146
I20260926 07:42:22.833155 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:35479
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:45311
--webserver_port=44959
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:22.838791 24214 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:42:22.843166 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Bootstrap starting.
I20260926 07:42:22.861846 24214 log.cc:824] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:22.870429 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:42:22.870946 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Bootstrap complete.
I20260926 07:42:22.879865 24214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:22.880745 24214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c688258aa94e049561d73d2250a2a1, State: Initialized, Role: FOLLOWER
I20260926 07:42:22.881400 24214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:22.881628 24214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:22.881739 24214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:22.881932 24214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:42:22.885658 24214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:22.886147 24214 leader_election.cc:302] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c688258aa94e049561d73d2250a2a1; no voters:
I20260926 07:42:22.886762 24214 leader_election.cc:288] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 07:42:22.886893 24218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:42:22.888074 24218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [term 2 LEADER]: Becoming Leader. State: Replica: 68c688258aa94e049561d73d2250a2a1, State: Running, Role: LEADER
I20260926 07:42:22.888693 24218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } }
I20260926 07:42:22.889024 24214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:22.891391 24219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68c688258aa94e049561d73d2250a2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } } }
I20260926 07:42:22.891812 24219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:22.894308 24220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c688258aa94e049561d73d2250a2a1. Latest consensus state: current_term: 2 leader_uuid: "68c688258aa94e049561d73d2250a2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c688258aa94e049561d73d2250a2a1" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45311 } } }
I20260926 07:42:22.894594 24220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:22.895991 24224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:22.902820 24224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:22.904606 24224 catalog_manager.cc:1259] Loaded cluster ID: e6b76197f6d34d108eb12dbf85777807
I20260926 07:42:22.904783 24224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:22.910084 24224 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:22.912057 24224 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Loaded TSK: 0
I20260926 07:42:22.914026 24224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:42:22.927917 24214 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 07:42:23.267027 24216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:23.267563 24216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:23.267772 24216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:23.279891 24216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:23.280028 24216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:23.280292 24216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:42:23.295675 24216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:35479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=44959
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:23.297336 24216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:23.299153 24216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:23.318518 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:23.319288 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:23.330363 24254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:23.331205 24216 server_base.cc:1034] running on GCE node
I20260926 07:42:23.332188 24216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:23.343519 24216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:23.344877 24216 hybrid_clock.cc:648] HybridClock initialized: now 1790408543344829 us; error 39 us; skew 500 ppm
I20260926 07:42:23.345376 24216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:23 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408543, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:23.349424 24216 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:42:23.393021 24216 webserver.cc:460] Webserver started at http://127.11.9.65:44959/ using document root <none> and password file <none>
I20260926 07:42:23.395110 24216 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:23.395364 24216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 07:42:23 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408543, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:23.436899 24216 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.005s sys 0.004s
I20260926 07:42:23.440747 24262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:23.442715 24216 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 07:42:23.442962 24216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "531b8db911e54c57a5616ecfd376b679"
format_stamp: "Formatted at 2026-09-26 07:42:20 on dist-test-slave-x492"
server_key: "74bf5e6a0e13435164e61398b7d5b55f"
server_key_iv: "befb2edff8e6a04875194388a17ee03e"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:23.444478 24216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:23.502122 24216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:23.507845 24216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:23.508169 24216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:23.509431 24216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:23.511344 24216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:23.511456 24216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:23.511566 24216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:23.511638 24216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:23.551216 24216 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:35479
I20260926 07:42:23.551311 24375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:35479 every 8 connection(s)
I20260926 07:42:23.552870 24216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:42:23.559713 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24216
I20260926 07:42:23.560483 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:38167
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:45311
--webserver_port=41369
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:23 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408543, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:23.588294 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:23.558649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.65:60653)
0926 07:42:23.560060 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:23.560106 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:23.562343 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:23.564793 (+ 2450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:23.564803 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:23.565447 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:23.566137 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:23.566171 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:23.568351 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:23.568356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:23.570181 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:23.570205 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:23.576429 (+ 6224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:23.576438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:23.576713 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:23.577780 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:23.577813 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:23.580222 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:23.583743 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:23.583752 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:23.583768 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:23.584186 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:23.584695 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:23.584700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:23.584704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:23.584923 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:23.585750 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:23.585758 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:23.586093 (+ 335us) client_negotiation.cc:769] Sending connection context
0926 07:42:23.586356 (+ 263us) client_negotiation.cc:240] Negotiation successful
0926 07:42:23.586914 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"thread_start_us":865,"threads_started":1}
I20260926 07:42:23.588294 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:23.558660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:60653 (local address 127.11.9.126:45311)
0926 07:42:23.559435 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:23.559455 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:23.560248 (+ 793us) server_negotiation.cc:407] Connection header received
0926 07:42:23.563068 (+ 2820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:23.563091 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:23.563575 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:23.563993 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:23.566380 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:23.568126 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:23.574632 (+ 6506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:23.575775 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:23.575958 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:23.580526 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:23.580534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:23.580551 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:23.580596 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:23.583594 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:23.584337 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:23.584346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:23.584353 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:23.584550 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:23.585048 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:23.585053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:23.585057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:23.585616 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:23.585819 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:23.586581 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 07:42:23.586957 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":2434,"threads_started":1}
I20260926 07:42:23.591878 24376 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:23.592206 24376 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:23.593087 24376 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
W20260926 07:42:23.596385 24376 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:45311 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 07:42:23.596616 24376 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:23.596916 24376 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:23.598258 24376 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:23.598543 24376 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
W20260926 07:42:23.599634 24376 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:42:23.978340 24380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:23.978798 24380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:23.978973 24380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:23.990293 24380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:23.990476 24380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:23.990685 24380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:42:24.006124 24380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:38167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=41369
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:24.008103 24380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:24.009894 24380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:24.033973 24396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:24.035225 24394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:24.036450 24400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:24.038918 24380 server_base.cc:1034] running on GCE node
I20260926 07:42:24.039757 24380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:24.057430 24380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:24.058823 24380 hybrid_clock.cc:648] HybridClock initialized: now 1790408544058705 us; error 124 us; skew 500 ppm
I20260926 07:42:24.059295 24380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:24.063638 24380 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:42:24.085578 24380 webserver.cc:460] Webserver started at http://127.11.9.66:41369/ using document root <none> and password file <none>
I20260926 07:42:24.086651 24380 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:24.086839 24380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:24.119017 24380 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 07:42:24.122125 24407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:24.123643 24380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:42:24.123831 24380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "fc39b69cbc6445f7a019515cd34c82ac"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "d785daab515796e5235fce09e2bc6ec7"
server_key_iv: "930298ef98b2bbd4e740a782e08a38be"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:24.124835 24380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:24.147054 24380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:24.152565 24380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:24.152961 24380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:24.157626 24380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:24.170172 24380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:24.170332 24380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:24.170540 24380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:24.170639 24380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:24.214030 24380 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:38167
I20260926 07:42:24.214057 24520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:38167 every 8 connection(s)
I20260926 07:42:24.215472 24380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:42:24.221480 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24380
I20260926 07:42:24.222446 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.9.84:48863/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:33535
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:45311
--webserver_port=42529
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:24.249050 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:24.223803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40709 (local address 127.11.9.126:45311)
0926 07:42:24.228712 (+ 4909us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:24.228720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:24.228743 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:42:24.228811 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:24.228818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:24.228907 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:24.231259 (+ 2352us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:24.231461 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.232738 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.235382 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.236150 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.236282 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:24.242043 (+ 5761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:24.242051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:24.242056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:24.242097 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:24.245811 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.246579 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.246585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.246589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.246709 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.247201 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.247208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.247210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.247498 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:24.247684 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:24.248446 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 07:42:24.248671 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4752,"thread_start_us":4638,"threads_started":1}
I20260926 07:42:24.250499 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:24.223172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.66:40709)
0926 07:42:24.224092 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:24.224142 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:24.227792 (+ 3650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:24.229356 (+ 1564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:24.229371 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:24.230168 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:24.231028 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:24.231068 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.232911 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.232920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:24.235183 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:24.235201 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.236601 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.236611 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:24.236943 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:24.238872 (+ 1929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:24.238910 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:24.241773 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:24.245972 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:24.245982 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:24.245999 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:24.246420 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:24.246832 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:24.246837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:24.246841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:24.247061 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:24.247645 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:24.247655 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:24.248057 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 07:42:24.248367 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 07:42:24.248918 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":1140,"threads_started":1}
I20260926 07:42:24.252192 24521 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:24.252526 24521 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:24.253407 24521 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:24.255317 24179 ts_manager.cc:194] Registered new tserver with Master: fc39b69cbc6445f7a019515cd34c82ac (127.11.9.66:38167)
I20260926 07:42:24.257673 24179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:40709
I20260926 07:42:24.599779 24376 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:24.600222 24376 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:24.601452 24179 ts_manager.cc:194] Registered new tserver with Master: 531b8db911e54c57a5616ecfd376b679 (127.11.9.65:35479)
I20260926 07:42:24.602562 24179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:60653
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
W20260926 07:42:24.694837 24525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:24.695225 24525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:24.695436 24525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:24.707165 24525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:24.707297 24525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:24.707532 24525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:42:24.721783 24525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.9.84:48863/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:33535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=42529
--tserver_master_addrs=127.11.9.126:45311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:24.723356 24525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:24.724653 24525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:24.741070 24530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:24.745244 24531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:24.745762 24533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:24.747095 24525 server_base.cc:1034] running on GCE node
I20260926 07:42:24.747720 24525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:24.748765 24525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:24.749995 24525 hybrid_clock.cc:648] HybridClock initialized: now 1790408544749936 us; error 49 us; skew 500 ppm
I20260926 07:42:24.750356 24525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:24.754117 24525 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:42:24.755915 24525 webserver.cc:460] Webserver started at http://127.11.9.67:42529/ using document root <none> and password file <none>
I20260926 07:42:24.756788 24525 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:24.756935 24525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:24.779091 24525 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 07:42:24.782625 24541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:24.784422 24525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 07:42:24.784614 24525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "bc484a8b1e9e4517b23c79420ce61463"
format_stamp: "Formatted at 2026-09-26 07:42:21 on dist-test-slave-x492"
server_key: "2e228b29800d3cf8bd7425dfe5ed43db"
server_key_iv: "0fe354efb0cb18267b021be0290ca4b0"
server_key_version: "kuduclusterkey@0"
I20260926 07:42:24.786007 24525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:24.815845 24525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:24.820919 24525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:24.821285 24525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:24.822491 24525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:24.824321 24525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:24.824466 24525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 07:42:24.824613 24525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:24.824721 24525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:24.921581 24654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:33535 every 8 connection(s)
I20260926 07:42:24.922425 24525 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:33535
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for HTTP/127.11.9.84@KRBTEST.COM
I20260926 07:42:24.924188 24525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:42:24.930488 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24525
Sep 26 07:42:24 dist-test-slave-x492 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408544, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:24.989890 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:24.932842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.11.9.126:45311)
0926 07:42:24.959778 (+ 26936us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:24.959785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:24.968824 (+ 9039us) server_negotiation.cc:407] Connection header received
0926 07:42:24.968984 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:24.968989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:24.969055 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:24.969182 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:24.971269 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.972413 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.974360 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.975007 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.975125 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:24.981385 (+ 6260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:24.981391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:24.981395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:24.981440 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:24.984702 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.986920 (+ 2218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.986925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.986929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.987068 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.988188 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.988192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.988195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.988531 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:24.988775 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:24.989283 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 07:42:24.989503 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26765,"thread_start_us":33011,"threads_started":1}
I20260926 07:42:24.995111 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:24.935233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:48549 (local address 127.11.9.126:45311)
0926 07:42:24.965821 (+ 30588us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:24.965830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:24.965860 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:42:24.966435 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:24.966442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:24.966555 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:24.966655 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:24.968732 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.970297 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.974106 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.975832 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.975939 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:24.980276 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:24.980282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:24.980287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:24.980322 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:24.986773 (+ 6451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.987968 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.987973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.987976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.988105 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:24.991092 (+ 2987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:24.991097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:24.991100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:24.991433 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:24.991646 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:24.994531 (+ 2885us) server_negotiation.cc:299] Negotiation successful
0926 07:42:24.994732 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30393,"thread_start_us":24581,"threads_started":1}
I20260926 07:42:24.996953 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:24.934751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45311 (local address 127.11.9.67:48549)
0926 07:42:24.962549 (+ 27798us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:24.962612 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:24.965045 (+ 2433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:24.967057 (+ 2012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:24.967069 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:24.967708 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:24.968497 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:24.968549 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.970486 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.970492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:24.973916 (+ 3424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:24.973942 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:24.976227 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:24.976234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:24.976505 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:24.977364 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:24.977394 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:24.980027 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:24.987206 (+ 7179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:24.987215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:24.987235 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:24.987826 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:24.988897 (+ 1071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:24.988902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:24.988906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:24.990631 (+ 1725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:24.991725 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:24.991733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:24.993365 (+ 1632us) client_negotiation.cc:769] Sending connection context
0926 07:42:24.993670 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 07:42:24.994265 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26232,"thread_start_us":26320,"threads_started":1}
I20260926 07:42:24.998854 24655 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45311
I20260926 07:42:24.999167 24655 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:25.000017 24655 heartbeater.cc:511] Master 127.11.9.126:45311 requested a full tablet report, sending...
I20260926 07:42:25.001665 24179 ts_manager.cc:194] Registered new tserver with Master: bc484a8b1e9e4517b23c79420ce61463 (127.11.9.67:33535)
I20260926 07:42:25.002732 24179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:48549
I20260926 07:42:25.009720 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:42:25.113519 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.091017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.11.9.126:45311)
0926 07:42:25.091237 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.091244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.095296 (+ 4052us) server_negotiation.cc:407] Connection header received
0926 07:42:25.095490 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.095496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.095610 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.095724 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:25.098189 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.099316 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.101479 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.102143 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.102248 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.105983 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:25.105990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:25.105995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:25.106036 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:25.110031 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.110947 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.110954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.110958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.111128 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.111705 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.111714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.111717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.112066 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:25.112302 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.112892 (+ 590us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.113123 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:42:25.120352 24179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:42:25.131999 24179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:42:25.190132 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.162669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:38167 (local address 127.0.0.1:55510)
0926 07:42:25.174779 (+ 12110us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.174822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:25.174971 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:25.175946 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:25.175955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:25.175984 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:25.176376 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.176391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.183326 (+ 6935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.183332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.186417 (+ 3085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.186442 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.188781 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.188789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.189087 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.189178 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 07:42:25.189339 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 07:42:25.189580 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11863,"mutex_wait_us":302,"thread_start_us":11736,"threads_started":1}
I20260926 07:42:25.191052 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.163228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.11.9.66:38167)
0926 07:42:25.175403 (+ 12175us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.175410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.175433 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:42:25.175512 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.175517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.175714 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.175939 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:25.176600 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.178212 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.186641 (+ 8429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.188229 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.188345 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.188551 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.190556 (+ 2005us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.190738 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12043,"thread_start_us":11942,"threads_started":1}
I20260926 07:42:25.191941 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.164571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56640)
0926 07:42:25.183208 (+ 18637us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.183232 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:25.183362 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:25.184389 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:25.184405 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:25.184427 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:25.184819 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.184832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.186844 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.186849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.189000 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.189015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.191173 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.191181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.191351 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.191384 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 07:42:25.191471 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 07:42:25.191611 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18483,"mutex_wait_us":8,"threads_started":1}
I20260926 07:42:25.192651 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.164899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.11.9.65:35479)
0926 07:42:25.182326 (+ 17427us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.182339 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.183500 (+ 1161us) server_negotiation.cc:407] Connection header received
0926 07:42:25.183613 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.183622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.183919 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.184145 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:25.185027 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.186660 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.189210 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.190656 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.190796 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.191016 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.192167 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.192325 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17195,"thread_start_us":17425,"threads_started":1}
I20260926 07:42:25.195940 24455 tablet_service.cc:1467] Processing CreateTablet for tablet 48726e8aeb7f42458fbe369571f93c9f (DEFAULT_TABLE table=test-table [id=638fe8f58b4044bdbe5703431cc3cb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:25.197265 24455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48726e8aeb7f42458fbe369571f93c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:25.200302 24311 tablet_service.cc:1467] Processing CreateTablet for tablet 48726e8aeb7f42458fbe369571f93c9f (DEFAULT_TABLE table=test-table [id=638fe8f58b4044bdbe5703431cc3cb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:25.202086 24311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48726e8aeb7f42458fbe369571f93c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:25.205358 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.182652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:33535 (local address 127.0.0.1:37630)
0926 07:42:25.191619 (+ 8967us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.191638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:25.191734 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:25.198530 (+ 6796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:25.198535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:25.198573 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:25.198727 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.198737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.200648 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.200653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.202496 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.202514 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.204605 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.204612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.204785 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.204823 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 07:42:25.204914 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 07:42:25.205052 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8877,"thread_start_us":653,"threads_started":1}
I20260926 07:42:25.206087 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.175811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.11.9.67:33535)
0926 07:42:25.193694 (+ 17883us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.193702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.193735 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:42:25.197774 (+ 4039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.197782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.198050 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.198227 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:25.198903 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.200476 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.202713 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.204168 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.204291 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.204499 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.205590 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.205747 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17704,"thread_start_us":17542,"threads_started":1}
I20260926 07:42:25.210021 24589 tablet_service.cc:1467] Processing CreateTablet for tablet 48726e8aeb7f42458fbe369571f93c9f (DEFAULT_TABLE table=test-table [id=638fe8f58b4044bdbe5703431cc3cb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:25.211548 24589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48726e8aeb7f42458fbe369571f93c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:25.219384 24676 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Bootstrap starting.
I20260926 07:42:25.224349 24676 tablet_bootstrap.cc:654] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:25.225744 24676 log.cc:824] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:25.235244 24676 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: No bootstrap required, opened a new log
I20260926 07:42:25.235579 24676 ts_tablet_manager.cc:1397] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20260926 07:42:25.245328 24680 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Bootstrap starting.
I20260926 07:42:25.247687 24676 raft_consensus.cc:348] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.248289 24676 raft_consensus.cc:374] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:25.248435 24676 raft_consensus.cc:729] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc39b69cbc6445f7a019515cd34c82ac, State: Initialized, Role: FOLLOWER
I20260926 07:42:25.249297 24676 consensus_queue.cc:260] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.250352 24681 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Bootstrap starting.
I20260926 07:42:25.254997 24680 tablet_bootstrap.cc:654] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:25.255560 24681 tablet_bootstrap.cc:654] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:25.256327 24680 log.cc:824] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:25.256913 24681 log.cc:824] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:25.259315 24521 heartbeater.cc:503] Master 127.11.9.126:45311 was elected leader, sending a full tablet report...
I20260926 07:42:25.260449 24676 ts_tablet_manager.cc:1428] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Time spent starting tablet: real 0.025s user 0.012s sys 0.008s
I20260926 07:42:25.264627 24681 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: No bootstrap required, opened a new log
I20260926 07:42:25.264911 24681 ts_tablet_manager.cc:1397] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20260926 07:42:25.272274 24680 tablet_bootstrap.cc:492] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: No bootstrap required, opened a new log
I20260926 07:42:25.272552 24680 ts_tablet_manager.cc:1397] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.010s
I20260926 07:42:25.273958 24681 raft_consensus.cc:348] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.274397 24681 raft_consensus.cc:374] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:25.274493 24681 raft_consensus.cc:729] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531b8db911e54c57a5616ecfd376b679, State: Initialized, Role: FOLLOWER
I20260926 07:42:25.275121 24681 consensus_queue.cc:260] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.276268 24376 heartbeater.cc:503] Master 127.11.9.126:45311 was elected leader, sending a full tablet report...
I20260926 07:42:25.276897 24681 ts_tablet_manager.cc:1428] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260926 07:42:25.282624 24680 raft_consensus.cc:348] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.282954 24680 raft_consensus.cc:374] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:25.283121 24680 raft_consensus.cc:729] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc484a8b1e9e4517b23c79420ce61463, State: Initialized, Role: FOLLOWER
I20260926 07:42:25.283731 24680 consensus_queue.cc:260] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.284741 24655 heartbeater.cc:503] Master 127.11.9.126:45311 was elected leader, sending a full tablet report...
I20260926 07:42:25.285389 24680 ts_tablet_manager.cc:1428] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260926 07:42:25.305692 24377 tablet.cc:2369] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:42:25.466279 24657 tablet.cc:2369] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:42:25.473781 24522 tablet.cc:2369] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:25.587111 24682 raft_consensus.cc:482] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:42:25.587431 24682 raft_consensus.cc:504] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.589301 24682 leader_election.cc:288] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 531b8db911e54c57a5616ecfd376b679 (127.11.9.65:35479), bc484a8b1e9e4517b23c79420ce61463 (127.11.9.67:33535)
I20260926 07:42:25.596853 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.589769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:58307 (local address 127.11.9.67:33535)
0926 07:42:25.589985 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.589993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.591244 (+ 1251us) server_negotiation.cc:407] Connection header received
0926 07:42:25.591411 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.591416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.591498 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.591596 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:25.592164 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.592900 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.595066 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.595823 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.595916 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.596051 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.596443 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.596584 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:42:25.596879 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.589874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:35533 (local address 127.11.9.65:35479)
0926 07:42:25.590082 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.590086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.590634 (+ 548us) server_negotiation.cc:407] Connection header received
0926 07:42:25.590946 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.590957 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.591048 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.591166 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:25.591945 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.592919 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.595006 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.595865 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.596026 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.596134 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.596493 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.596626 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:42:25.597065 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.590298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:33535 (local address 127.11.9.66:58307)
0926 07:42:25.591130 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.591147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:25.591271 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:25.591620 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:25.591623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:25.591642 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:25.592006 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.592020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.593033 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.593038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.594879 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.594891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.596033 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.596051 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.596252 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.596297 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 07:42:25.596385 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 07:42:25.596550 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":79,"thread_start_us":533,"threads_started":1}
I20260926 07:42:25.597091 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.589788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.11.9.66:35533)
0926 07:42:25.590502 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.590524 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:25.590698 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:25.591232 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:25.591237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:25.591302 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:25.591802 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.591813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.593077 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.593086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.594857 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:25.594868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.596142 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.596146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:25.596298 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.596320 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:42:25.596445 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 07:42:25.596581 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":60,"spinlock_wait_cycles":29568,"thread_start_us":700,"threads_started":1}
I20260926 07:42:25.598156 24609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48726e8aeb7f42458fbe369571f93c9f" candidate_uuid: "fc39b69cbc6445f7a019515cd34c82ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc484a8b1e9e4517b23c79420ce61463" is_pre_election: true
I20260926 07:42:25.598469 24331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48726e8aeb7f42458fbe369571f93c9f" candidate_uuid: "fc39b69cbc6445f7a019515cd34c82ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "531b8db911e54c57a5616ecfd376b679" is_pre_election: true
I20260926 07:42:25.598680 24609 raft_consensus.cc:2454] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc39b69cbc6445f7a019515cd34c82ac in term 0.
I20260926 07:42:25.599006 24331 raft_consensus.cc:2454] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc39b69cbc6445f7a019515cd34c82ac in term 0.
I20260926 07:42:25.599337 24411 leader_election.cc:302] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc484a8b1e9e4517b23c79420ce61463, fc39b69cbc6445f7a019515cd34c82ac; no voters:
I20260926 07:42:25.599786 24682 raft_consensus.cc:2781] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:42:25.599946 24682 raft_consensus.cc:482] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:42:25.600056 24682 raft_consensus.cc:3037] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:25.602298 24682 raft_consensus.cc:504] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.602885 24682 leader_election.cc:288] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [CANDIDATE]: Term 1 election: Requested vote from peers 531b8db911e54c57a5616ecfd376b679 (127.11.9.65:35479), bc484a8b1e9e4517b23c79420ce61463 (127.11.9.67:33535)
I20260926 07:42:25.603314 24331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48726e8aeb7f42458fbe369571f93c9f" candidate_uuid: "fc39b69cbc6445f7a019515cd34c82ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "531b8db911e54c57a5616ecfd376b679"
I20260926 07:42:25.603314 24609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48726e8aeb7f42458fbe369571f93c9f" candidate_uuid: "fc39b69cbc6445f7a019515cd34c82ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc484a8b1e9e4517b23c79420ce61463"
I20260926 07:42:25.603518 24331 raft_consensus.cc:3037] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:25.603518 24609 raft_consensus.cc:3037] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:25.606601 24609 raft_consensus.cc:2454] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc39b69cbc6445f7a019515cd34c82ac in term 1.
I20260926 07:42:25.606601 24331 raft_consensus.cc:2454] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc39b69cbc6445f7a019515cd34c82ac in term 1.
I20260926 07:42:25.606978 24411 leader_election.cc:302] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc484a8b1e9e4517b23c79420ce61463, fc39b69cbc6445f7a019515cd34c82ac; no voters:
I20260926 07:42:25.607273 24682 raft_consensus.cc:2781] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:25.607738 24682 raft_consensus.cc:686] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 1 LEADER]: Becoming Leader. State: Replica: fc39b69cbc6445f7a019515cd34c82ac, State: Running, Role: LEADER
I20260926 07:42:25.608260 24682 consensus_queue.cc:237] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } }
I20260926 07:42:25.612859 24179 catalog_manager.cc:5495] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac reported cstate change: term changed from 0 to 1, leader changed from <none> to fc39b69cbc6445f7a019515cd34c82ac (127.11.9.66). New cstate: current_term: 1 leader_uuid: "fc39b69cbc6445f7a019515cd34c82ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc39b69cbc6445f7a019515cd34c82ac" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 38167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 } health_report { overall_health: UNKNOWN } } }
I20260926 07:42:25.664716 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.659117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.11.9.66:38167)
0926 07:42:25.659316 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.659321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.659644 (+ 323us) server_negotiation.cc:407] Connection header received
0926 07:42:25.659803 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.659806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.659900 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.660020 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:42:25.660419 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.661134 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.662587 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.662992 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.663084 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.663530 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:42:25.664048 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:42:25.664158 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.664307 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.664414 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:42:25.673074 24609 raft_consensus.cc:1261] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 1 FOLLOWER]: Refusing update from remote peer fc39b69cbc6445f7a019515cd34c82ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:25.673066 24331 raft_consensus.cc:1261] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 1 FOLLOWER]: Refusing update from remote peer fc39b69cbc6445f7a019515cd34c82ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:25.674173 24682 consensus_queue.cc:1035] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc484a8b1e9e4517b23c79420ce61463" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:42:25.674470 24694 consensus_queue.cc:1035] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "531b8db911e54c57a5616ecfd376b679" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:42:25.685617 24698 mvcc.cc:205] Tried to move back new op lower bound from 7333513403066716160 to 7333513402815995904. Current Snapshot: MvccSnapshot[applied={T|T < 7333513403066716160}]
I20260926 07:42:25.698076 24179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:25.698410 24179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:25.701742 24179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68c688258aa94e049561d73d2250a2a1: Sending DeleteTablet for 3 replicas of tablet 48726e8aeb7f42458fbe369571f93c9f
I20260926 07:42:25.702904 24455 tablet_service.cc:1514] Processing DeleteTablet for tablet 48726e8aeb7f42458fbe369571f93c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:55510
I20260926 07:42:25.702904 24311 tablet_service.cc:1514] Processing DeleteTablet for tablet 48726e8aeb7f42458fbe369571f93c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:56640
I20260926 07:42:25.703649 24589 tablet_service.cc:1514] Processing DeleteTablet for tablet 48726e8aeb7f42458fbe369571f93c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:37630
I20260926 07:42:25.704164 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24216
I20260926 07:42:25.706954 24709 tablet_replica.cc:330] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: stopping tablet replica
I20260926 07:42:25.707446 24709 raft_consensus.cc:2229] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:25.708396 24709 raft_consensus.cc:2258] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:25.713536 24709 ts_tablet_manager.cc:1905] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:25.717386 24707 tablet_replica.cc:330] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: stopping tablet replica
I20260926 07:42:25.718005 24708 tablet_replica.cc:330] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: stopping tablet replica
I20260926 07:42:25.718504 24708 raft_consensus.cc:2229] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:42:25.719024 24707 raft_consensus.cc:2229] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:25.719763 24708 raft_consensus.cc:2258] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:25.720302 24707 raft_consensus.cc:2258] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:25.721148 24709 ts_tablet_manager.cc:1918] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:25.721387 24709 log.cc:1196] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/48726e8aeb7f42458fbe369571f93c9f
I20260926 07:42:25.722021 24709 ts_tablet_manager.cc:1939] T 48726e8aeb7f42458fbe369571f93c9f P bc484a8b1e9e4517b23c79420ce61463: Deleting consensus metadata
I20260926 07:42:25.722847 24707 ts_tablet_manager.cc:1905] T 48726e8aeb7f42458fbe369571f93c9f P 531b8db911e54c57a5616ecfd376b679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:25.723968 24166 catalog_manager.cc:4841] TS bc484a8b1e9e4517b23c79420ce61463 (127.11.9.67:33535): tablet 48726e8aeb7f42458fbe369571f93c9f (table test-table [id=638fe8f58b4044bdbe5703431cc3cb9c]) successfully deleted
I20260926 07:42:25.724541 24708 ts_tablet_manager.cc:1905] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:25.726367 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.705350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.11.9.65:35479)
0926 07:42:25.705542 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.705565 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.709188 (+ 3623us) server_negotiation.cc:407] Connection header received
0926 07:42:25.709403 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.709408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.709501 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.709619 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:25.711237 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.712034 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.713968 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.714634 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.714730 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.718751 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:25.718769 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:25.718788 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:25.718826 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:25.722850 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.723737 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.723745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.723748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.723915 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.724399 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.724404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.724407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.724917 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:25.725150 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.725746 (+ 596us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.725999 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 07:42:25.729532 24371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:25.731958 24708 ts_tablet_manager.cc:1918] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:25.732355 24708 log.cc:1196] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/48726e8aeb7f42458fbe369571f93c9f
I20260926 07:42:25.733378 24708 ts_tablet_manager.cc:1939] T 48726e8aeb7f42458fbe369571f93c9f P fc39b69cbc6445f7a019515cd34c82ac: Deleting consensus metadata
I20260926 07:42:25.735291 24166 catalog_manager.cc:4841] TS fc39b69cbc6445f7a019515cd34c82ac (127.11.9.66:38167): tablet 48726e8aeb7f42458fbe369571f93c9f (table test-table [id=638fe8f58b4044bdbe5703431cc3cb9c]) successfully deleted
I20260926 07:42:25.820652 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24216
I20260926 07:42:25.838315 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24380
W20260926 07:42:25.838289 24164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:35479 (error 108)
I20260926 07:42:25.853449 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.840818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56662)
0926 07:42:25.853019 (+ 12201us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.853197 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12039,"thread_start_us":11921,"threads_started":1}
W20260926 07:42:25.853961 24164 catalog_manager.cc:4584] TS 531b8db911e54c57a5616ecfd376b679 (127.11.9.65:35479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48726e8aeb7f42458fbe369571f93c9f: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
I20260926 07:42:25.856426 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.839525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.11.9.66:38167)
0926 07:42:25.839889 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.839895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.839913 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:42:25.839975 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.839981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.840056 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.840170 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:25.841877 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.843075 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.844934 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.845605 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.845713 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:25.848954 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:25.848974 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:25.848998 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:25.849046 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:25.853082 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.853994 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.854003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.854006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.854169 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:25.854615 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:25.854621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:25.854624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:25.854991 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:25.855371 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:25.855686 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 07:42:25.856022 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260926 07:42:25.857544 24515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:25.907927 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.907411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56676)
0926 07:42:25.907655 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.907766 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 07:42:25.909116 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.908659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56682)
0926 07:42:25.908867 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.908944 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 07:42:25.969599 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.969056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56698)
0926 07:42:25.969306 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.969410 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 07:42:25.970780 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.970333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56712)
0926 07:42:25.970487 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:25.970584 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 07:42:25.972204 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24380
I20260926 07:42:25.993538 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24525
I20260926 07:42:26.007272 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:25.994408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.11.9.67:33535)
0926 07:42:25.994581 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:25.994589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:25.994612 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:42:25.994859 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:25.994863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:25.994939 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:25.995051 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:25.996355 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.997103 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.998568 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:25.998964 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:25.999126 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:26.001655 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:26.001669 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:26.001684 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:26.001719 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:26.004411 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:26.005094 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:26.005107 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:26.005110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:26.005306 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:26.005682 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:26.005688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:26.005692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:26.006042 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:26.006370 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:26.006685 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 07:42:26.006891 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 07:42:26.008035 24650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:26.051676 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:26.051159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56714)
0926 07:42:26.051439 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:26.051525 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 07:42:26.052795 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:26.052393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:35479 (local address 127.0.0.1:56724)
0926 07:42:26.052598 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:26.052677 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:35479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 07:42:26.096154 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24525
I20260926 07:42:26.113562 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24146
I20260926 07:42:26.114792 24209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:26.226538 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24146
I20260926 07:42:26.247970 11301 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 07:42:26.583009 11301 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 07:42:26.583299 11301 mini_ranger.cc:66] Stopping Ranger...
I20260926 07:42:26.928423 11301 mini_ranger.cc:68] Stopped Ranger
2026-09-26 07:42:26.928 UTC [23198] LOG: received smart shutdown request
2026-09-26 07:42:26.931 UTC [23198] LOG: background worker "logical replication launcher" (PID 23207) exited with exit code 1
2026-09-26 07:42:26.932 UTC [23202] LOG: shutting down
2026-09-26 07:42:27.128 UTC [23198] LOG: database system is shut down
2026-09-26T07:42:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52394 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:42:27 dist-test-slave-x492 krb5kdc[24725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:42:27 dist-test-slave-x492 krb5kdc[24725](info): set up 2 sockets
Sep 26 07:42:27 dist-test-slave-x492 krb5kdc[24725](info): commencing operation
krb5kdc: starting...
W20260926 07:42:28.353713 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.009s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:42:28 dist-test-slave-x492 krb5kdc[24725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:42:28.577014 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35101
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:28.901299 24738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:28.901572 24738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:28.901661 24738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:28.908634 24738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:28.908733 24738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:28.908777 24738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:28.908835 24738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:28.908878 24738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:28.918496 24738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:28.919701 24738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:28.920946 24738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:28.932448 24743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:28.932825 24744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:28.932847 24746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:28.933820 24738 server_base.cc:1034] running on GCE node
I20260926 07:42:28.934615 24738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:28.935967 24738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:28.937255 24738 hybrid_clock.cc:648] HybridClock initialized: now 1790408548937221 us; error 45 us; skew 500 ppm
I20260926 07:42:28.937611 24738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:28 dist-test-slave-x492 krb5kdc[24725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:28.940613 24738 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:28.942180 24738 webserver.cc:460] Webserver started at http://127.11.9.126:34929/ using document root <none> and password file <none>
I20260926 07:42:28.942798 24738 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:28.942893 24738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:28.943137 24738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:28.946003 24738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "ba2c56886413432aa5025651527cdeb9"
format_stamp: "Formatted at 2026-09-26 07:42:28 on dist-test-slave-x492"
server_key: "722065331d46a112a2481e99fc9574a8"
server_key_iv: "5876b6208d5879917f2831c4779f235c"
server_key_version: "encryptionkey@0"
I20260926 07:42:28.946728 24738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "ba2c56886413432aa5025651527cdeb9"
format_stamp: "Formatted at 2026-09-26 07:42:28 on dist-test-slave-x492"
server_key: "722065331d46a112a2481e99fc9574a8"
server_key_iv: "5876b6208d5879917f2831c4779f235c"
server_key_version: "encryptionkey@0"
I20260926 07:42:28.951659 24738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:42:28.955123 24753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:28.956544 24738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:42:28.956748 24738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "ba2c56886413432aa5025651527cdeb9"
format_stamp: "Formatted at 2026-09-26 07:42:28 on dist-test-slave-x492"
server_key: "722065331d46a112a2481e99fc9574a8"
server_key_iv: "5876b6208d5879917f2831c4779f235c"
server_key_version: "encryptionkey@0"
I20260926 07:42:28.956984 24738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:28.976184 24738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:28.980561 24738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:28.980875 24738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:29.000332 24738 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:35101
I20260926 07:42:29.000375 24804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:35101 every 8 connection(s)
I20260926 07:42:29.001763 24738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:29.005527 24805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:29.008972 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24738
I20260926 07:42:29.009224 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:42:29.010061 11301 external_mini_cluster.cc:1428] Setting key 580a4f19376c8b38886234b3d6bf5e82
I20260926 07:42:29.014737 24805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.125@KRBTEST.COM" created.
I20260926 07:42:29.039228 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.019027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.11.9.126:35101)
0926 07:42:29.019953 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.019973 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.020428 (+ 455us) server_negotiation.cc:407] Connection header received
0926 07:42:29.022534 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.022539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.022777 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.023005 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.024314 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.025587 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.026988 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.027665 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.027872 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.031547 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.031555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.031566 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.031598 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:29.034832 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.035512 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.035520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.035524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.035681 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.036094 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.036102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.036109 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.036600 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:29.036825 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:29.037397 (+ 572us) server_negotiation.cc:299] Negotiation successful
0926 07:42:29.037756 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":651,"thread_start_us":433,"threads_started":1}
I20260926 07:42:29.039284 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.019028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35101 (local address 127.0.0.1:36018)
0926 07:42:29.020251 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.020294 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:29.022146 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:29.023015 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:29.023022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:29.023746 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:29.024088 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.024105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.025755 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.025760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.026837 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.026849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.027903 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.027909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.028077 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.028803 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:29.028824 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:29.031328 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:29.034971 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.034984 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.034995 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.035374 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.035798 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.035805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.035808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.035981 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.036758 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:29.036766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:29.036987 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 07:42:29.037296 (+ 309us) client_negotiation.cc:240] Negotiation successful
0926 07:42:29.037866 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":149,"thread_start_us":783,"threads_started":1}
I20260926 07:42:29.042488 24805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:42:29.043646 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.043232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51194)
0926 07:42:29.043429 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.043537 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 07:42:29.043965 24757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
I20260926 07:42:29.045593 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.045268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51200)
0926 07:42:29.045421 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.045487 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 07:42:29.045783 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:35641: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
I20260926 07:42:29.060623 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:35641
--webserver_interface=127.11.9.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:29.095103 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } attempt: 1
I20260926 07:42:29.096274 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.095821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51212)
0926 07:42:29.096024 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.096120 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 07:42:29.097414 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.096997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51220)
0926 07:42:29.097227 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.097291 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 07:42:29.097780 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:35641: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
I20260926 07:42:29.166049 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } attempt: 2
I20260926 07:42:29.167192 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.166719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51226)
0926 07:42:29.166923 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.167051 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 07:42:29.168227 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.167837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51232)
0926 07:42:29.168007 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.168098 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260926 07:42:29.168424 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:35641: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
I20260926 07:42:29.259693 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } attempt: 3
I20260926 07:42:29.260874 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.260412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51242)
0926 07:42:29.260622 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.260753 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 07:42:29.261992 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.261658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51248)
0926 07:42:29.261801 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.261872 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 07:42:29.262214 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:35641: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
W20260926 07:42:29.385442 24812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:29.385703 24812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:29.385860 24812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:29.393074 24812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:29.393185 24812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:29.393263 24812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:29.393373 24812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:29.393440 24812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:29.403028 24812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.9.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:29.404276 24812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:29.405478 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } attempt: 4
I20260926 07:42:29.405586 24812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 07:42:29.406668 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.406165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51264)
0926 07:42:29.406468 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.406549 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260926 07:42:29.407696 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.407378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51266)
0926 07:42:29.407510 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.407583 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 07:42:29.407948 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:35641: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
W20260926 07:42:29.418213 24817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:29.418128 24820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:29.418179 24818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:29.419636 24812 server_base.cc:1034] running on GCE node
I20260926 07:42:29.420397 24812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:29.421727 24812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:29.422998 24812 hybrid_clock.cc:648] HybridClock initialized: now 1790408549422957 us; error 57 us; skew 500 ppm
I20260926 07:42:29.423347 24812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:29.426481 24812 init.cc:377] Logged in from keytab as kudu/127.11.9.125@KRBTEST.COM (short username kudu)
I20260926 07:42:29.427965 24812 webserver.cc:460] Webserver started at http://127.11.9.125:46729/ using document root <none> and password file <none>
I20260926 07:42:29.428635 24812 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:29.428782 24812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:29.429093 24812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:29.432051 24812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data/instance:
uuid: "3dc55d6043ae47a3aa947e5e469652dc"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "b97576d0fb12f719994ed2fd431c9e27"
server_key_iv: "083b1cba0eeb767a5c46e8048a3a1b74"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.432848 24812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal/instance:
uuid: "3dc55d6043ae47a3aa947e5e469652dc"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "b97576d0fb12f719994ed2fd431c9e27"
server_key_iv: "083b1cba0eeb767a5c46e8048a3a1b74"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.438082 24812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260926 07:42:29.441850 24827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:29.443239 24812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:42:29.443502 24812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "3dc55d6043ae47a3aa947e5e469652dc"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "b97576d0fb12f719994ed2fd431c9e27"
server_key_iv: "083b1cba0eeb767a5c46e8048a3a1b74"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.443720 24812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:29.458390 24812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:29.463230 24812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:29.463551 24812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:29.483795 24812 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.125:35641
I20260926 07:42:29.483827 24878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.125:35641 every 8 connection(s)
I20260926 07:42:29.485366 24812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
I20260926 07:42:29.489051 24879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:29.491277 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24812
I20260926 07:42:29.491484 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-1/wal/instance
I20260926 07:42:29.491950 11301 external_mini_cluster.cc:1428] Setting key 935f5cfad138dd33b364f8d76936b40d
I20260926 07:42:29.495888 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.124@KRBTEST.COM" created.
I20260926 07:42:29.516459 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.499133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.11.9.126:35101)
0926 07:42:29.499327 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.499331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.500616 (+ 1285us) server_negotiation.cc:407] Connection header received
0926 07:42:29.502544 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.502551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.502654 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.502775 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.504233 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.504902 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.506506 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.506954 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.507101 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.510503 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.510508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.510512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.510545 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:29.513354 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.514157 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.514163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.514168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.514288 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.514746 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.514750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.514754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.515057 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:29.515225 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:29.515920 (+ 695us) server_negotiation.cc:299] Negotiation successful
0926 07:42:29.516136 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:42:29.517519 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.499672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35101 (local address 127.0.0.1:36028)
0926 07:42:29.500435 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.500476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:29.502260 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:29.503004 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:29.503012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:29.503504 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:29.504032 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.504047 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.505079 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.505084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.506349 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.506361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.507197 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.507204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.507419 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.508146 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:29.508167 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:29.510270 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:29.513500 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.513509 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.513524 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.514017 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.514417 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.514422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.514425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.514617 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.515199 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:29.515206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:29.515543 (+ 337us) client_negotiation.cc:769] Sending connection context
0926 07:42:29.515808 (+ 265us) client_negotiation.cc:240] Negotiation successful
0926 07:42:29.516317 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":354,"threads_started":1}
I20260926 07:42:29.519207 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:42:29.536433 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.519998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51274)
0926 07:42:29.520177 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.520197 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:29.520296 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:29.521542 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:29.521545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:29.521914 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:29.522043 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.522052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.524138 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.524146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.525736 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.525754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.526911 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.526923 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.527164 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.528062 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:29.528088 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:29.530571 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:29.533898 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.533906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.533910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.534273 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.534656 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.534662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.534668 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.534816 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.535427 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:29.535432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:29.535568 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 07:42:29.535727 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 07:42:29.535996 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 07:42:29.536474 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.520305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.11.9.125:35641)
0926 07:42:29.521004 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.521015 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.521048 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:42:29.521107 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.521117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.521379 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.521536 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.522204 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.523983 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.525911 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.526745 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.526861 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.530785 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.530796 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.530808 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.530834 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:29.533765 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.534409 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.534417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.534421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.534533 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.534934 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.534942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.534945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.535287 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:29.535510 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:29.535886 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 07:42:29.536087 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":370,"threads_started":1}
I20260926 07:42:29.539069 24879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:42:29.540203 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.539674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42592)
0926 07:42:29.539862 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.540019 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260926 07:42:29.540488 24829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.542119 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.541756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42606)
0926 07:42:29.541934 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.542024 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 07:42:29.542419 24879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
I20260926 07:42:29.544730 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:39185
--webserver_interface=127.11.9.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:29.591810 24879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 1
I20260926 07:42:29.593058 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.592604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42622)
0926 07:42:29.592810 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.592916 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 07:42:29.594086 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.593696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42636)
0926 07:42:29.593863 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.593954 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 07:42:29.594286 24879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.662498 24879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 2
I20260926 07:42:29.663625 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.663173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42642)
0926 07:42:29.663374 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.663468 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 07:42:29.664623 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.664274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42644)
0926 07:42:29.664435 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.664506 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 07:42:29.664831 24879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.707394 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } attempt: 5
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:42:29.720795 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.708228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51284)
0926 07:42:29.708433 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.708472 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:29.708635 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:29.709013 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:29.709018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:29.709531 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:29.709738 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.709746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.710775 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.710781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.711646 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.711657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.712454 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.712460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.712651 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.713331 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:29.713349 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:29.715329 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:29.718415 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.718422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.718426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.718783 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.719166 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:29.719171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:29.719175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:29.719317 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:29.719850 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:29.719855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:29.720014 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 07:42:29.720167 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 07:42:29.720444 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 07:42:29.720796 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.708418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.11.9.125:35641)
0926 07:42:29.708633 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.708638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.708659 (+ 21us) server_negotiation.cc:407] Connection header received
0926 07:42:29.708752 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.708757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.708846 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.708964 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.709926 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.710619 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.711799 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.712295 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.712442 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.715549 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.715554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.715557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.715587 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:29.718272 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.718912 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.718919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.718923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.719043 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:29.719436 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:29.719442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:29.719446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:29.719722 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:29.719882 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:29.720261 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 07:42:29.720475 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 07:42:29.722378 24805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:42:29.723412 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.723038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42652)
0926 07:42:29.723201 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.723276 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 07:42:29.724337 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.724060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42666)
0926 07:42:29.724181 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.724255 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 07:42:29.724611 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.756049 24879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 3
I20260926 07:42:29.757249 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.756680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42678)
0926 07:42:29.756919 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.757010 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 07:42:29.758320 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.757915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42688)
0926 07:42:29.758137 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.758217 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260926 07:42:29.758522 24879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.775844 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 1
I20260926 07:42:29.777050 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.776567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42702)
0926 07:42:29.776758 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.776860 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 07:42:29.778060 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.777715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42704)
0926 07:42:29.777892 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.777960 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 07:42:29.778273 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.846542 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 2
I20260926 07:42:29.847857 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.847276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42706)
0926 07:42:29.847497 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.847666 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 07:42:29.848918 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.848569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42708)
0926 07:42:29.848723 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.848801 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 07:42:29.849189 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
W20260926 07:42:29.876308 24886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:29.876578 24886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:29.876740 24886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:29.883899 24886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:29.883998 24886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:29.884099 24886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:29.884205 24886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:29.884267 24886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:29.894496 24886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35101,127.11.9.125:35641,127.11.9.124:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.9.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:29.895803 24886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:29.897091 24886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 07:42:29.901741 24879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 4
I20260926 07:42:29.902967 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.902423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42716)
0926 07:42:29.902652 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.902763 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 07:42:29.904196 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.903766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42730)
0926 07:42:29.904010 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.904089 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260926 07:42:29.904490 24879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
W20260926 07:42:29.909821 24894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:29.909940 24892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:29.909982 24891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:29.910985 24886 server_base.cc:1034] running on GCE node
I20260926 07:42:29.911772 24886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:29.913090 24886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:29.914359 24886 hybrid_clock.cc:648] HybridClock initialized: now 1790408549914306 us; error 37 us; skew 500 ppm
I20260926 07:42:29.914721 24886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:29.918058 24886 init.cc:377] Logged in from keytab as kudu/127.11.9.124@KRBTEST.COM (short username kudu)
I20260926 07:42:29.919524 24886 webserver.cc:460] Webserver started at http://127.11.9.124:37401/ using document root <none> and password file <none>
I20260926 07:42:29.920197 24886 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:29.920363 24886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:29.920658 24886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:29.923623 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data/instance:
uuid: "215b0df3b1fc4a9dbe66886785781919"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "05d8d94a9fdc72aeb9a504cb7bc80fcb"
server_key_iv: "c81c5cd83c7a5a36adc1a81aa06d65ce"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.924403 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal/instance:
uuid: "215b0df3b1fc4a9dbe66886785781919"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "05d8d94a9fdc72aeb9a504cb7bc80fcb"
server_key_iv: "c81c5cd83c7a5a36adc1a81aa06d65ce"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.929585 24886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 07:42:29.933254 24901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:29.934870 24886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:42:29.935079 24886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "215b0df3b1fc4a9dbe66886785781919"
format_stamp: "Formatted at 2026-09-26 07:42:29 on dist-test-slave-x492"
server_key: "05d8d94a9fdc72aeb9a504cb7bc80fcb"
server_key_iv: "c81c5cd83c7a5a36adc1a81aa06d65ce"
server_key_version: "encryptionkey@0"
I20260926 07:42:29.935281 24886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:29.947821 24886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:29.951658 24886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:29.951941 24886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:29.955443 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 3
I20260926 07:42:29.956602 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.956105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42736)
0926 07:42:29.956360 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.956443 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 07:42:29.957916 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.957377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42742)
0926 07:42:29.957650 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.957754 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260926 07:42:29.958165 24805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39185: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39185: connect: Connection refused (error 111)
I20260926 07:42:29.970872 24886 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.124:39185
I20260926 07:42:29.970901 24952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.124:39185 every 8 connection(s)
I20260926 07:42:29.972256 24886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
I20260926 07:42:29.976105 24953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:29.976640 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 24886
I20260926 07:42:29.976852 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790408351541924-11301-0/minicluster-data/master-2/wal/instance
I20260926 07:42:29.977536 11301 external_mini_cluster.cc:1428] Setting key 2ff2f360b5f65884938f2ee151e225e1
I20260926 07:42:29.986174 24953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:42:29 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:30.004470 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.988658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.11.9.126:35101)
0926 07:42:29.988873 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.988881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.989668 (+ 787us) server_negotiation.cc:407] Connection header received
0926 07:42:29.989850 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.989857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.989939 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.990061 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.991055 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.991742 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.993450 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.994200 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.994339 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.997599 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.997606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.997611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.997648 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.001310 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.002130 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.002135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.002138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.002306 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.002721 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.002725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.002729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.003030 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.003166 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.003810 (+ 644us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.004031 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:42:30.005327 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.980911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.11.9.124:39185)
0926 07:42:29.983928 (+ 3017us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:29.983945 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:29.983988 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:42:29.985718 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:29.985741 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:29.986132 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:29.986557 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:29.987990 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.989559 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.991174 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.991774 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.992007 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.996328 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:29.996335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:29.996354 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:29.996381 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:29.999990 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.000655 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.000663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.000676 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.000816 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.001325 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.001332 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.001336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.002046 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.002278 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.003494 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.003961 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":2784,"thread_start_us":2367,"threads_started":1}
I20260926 07:42:30.005393 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:29.988651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35101 (local address 127.0.0.1:36036)
0926 07:42:29.989521 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:29.989561 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:29.989710 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:29.990105 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:29.990110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:29.990630 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:29.990886 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.990895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.991922 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.991927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.993310 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:29.993322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:29.994442 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:29.994447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:29.994556 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:29.995144 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:29.995165 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:29.997378 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:30.001439 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.001451 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.001458 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.001996 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.002426 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.002430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.002433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.002595 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.003174 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:30.003182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:30.003494 (+ 312us) client_negotiation.cc:769] Sending connection context
0926 07:42:30.003665 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 07:42:30.004085 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":135,"spinlock_wait_cycles":17664,"thread_start_us":461,"threads_started":1}
I20260926 07:42:30.007004 24953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:30.022168 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.007876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51300)
0926 07:42:30.008358 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:30.008377 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:30.008492 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:30.009034 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:30.009040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:30.009460 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:30.009594 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.009602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.010571 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.010579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.011967 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.011980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.012924 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.012930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.013152 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.013735 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:30.013757 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:30.015987 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:30.019660 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.019663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.019666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.020036 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.020511 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.020515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.020526 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.020668 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.021302 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:30.021306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:30.021432 (+ 126us) client_negotiation.cc:769] Sending connection context
0926 07:42:30.021566 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 07:42:30.021768 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355}
I20260926 07:42:30.022218 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.008153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.11.9.125:35641)
0926 07:42:30.008365 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.008371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.008448 (+ 77us) server_negotiation.cc:407] Connection header received
0926 07:42:30.008654 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.008661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.008739 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.008814 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.009752 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.010424 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.012111 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.012555 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.012713 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.016183 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.016189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.016193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.016231 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.019525 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.020161 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.020169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.020172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.020388 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.020787 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.020793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.020799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.021134 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.021311 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.021697 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.021903 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 07:42:30.023459 24953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:42:30.024883 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.008746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.11.9.126:35101)
0926 07:42:30.008896 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.008901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.008916 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:42:30.008966 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.008970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.009035 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.009130 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.010678 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.011800 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.013112 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.013745 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.013889 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.018641 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.018648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.018654 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.018692 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.021853 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.022638 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.022642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.022645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.022782 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.023220 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.023225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.023227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.023709 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.024042 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.024313 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.024557 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:42:30.037678 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.024185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42764)
0926 07:42:30.024368 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:30.024386 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:30.024484 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:30.024878 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:30.024881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:30.025394 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:30.025572 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.025580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.026606 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.026614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.028069 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.028086 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.028836 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.028840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.029044 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.029668 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:30.029684 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:30.032136 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:30.035215 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.035222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.035229 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.035588 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.036016 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.036020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.036024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.036193 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.036726 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:30.036733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:30.036905 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 07:42:30.037024 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 07:42:30.037313 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 07:42:30.037909 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.024404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.11.9.124:39185)
0926 07:42:30.024555 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.024560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.024577 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:42:30.024622 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.024627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.024691 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.024783 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.025716 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.026476 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.028219 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.028704 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.028839 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.032309 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.032313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.032316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.032342 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.035073 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.035716 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.035723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.035730 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.035900 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.036343 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.036347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.036349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.036597 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.036703 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.037193 (+ 490us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.037489 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":51}
I20260926 07:42:30.042580 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.026761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.11.9.125:35641)
0926 07:42:30.026952 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.026956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.026970 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:42:30.027071 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.027074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.027123 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.027198 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.028704 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.029652 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.031370 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.031963 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.032059 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.036870 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.036874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.036878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.036907 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.040015 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.040641 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.040647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.040650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.040803 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.041211 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.041216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.041219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.041541 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.041811 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.042033 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.042248 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:42:30.044111 24953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: Bootstrap starting.
I20260926 07:42:30.047667 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:42:30.048130 24953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:30.049415 24953 log.cc:824] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:30.052716 24953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: No bootstrap required, opened a new log
I20260926 07:42:30.064306 24953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.064805 24953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:30.064924 24953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215b0df3b1fc4a9dbe66886785781919, State: Initialized, Role: FOLLOWER
I20260926 07:42:30.065809 24953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.075075 24953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:30.075754 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.053419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.11.9.125:35641)
0926 07:42:30.053602 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.053608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.054159 (+ 551us) server_negotiation.cc:407] Connection header received
0926 07:42:30.054438 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.054445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.054527 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.054598 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.057203 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.058321 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.060751 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.061410 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.061529 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.066275 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.066282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.066287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.066328 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.071802 (+ 5474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.072855 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.072863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.072868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.073031 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.073542 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.073547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.073550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.073959 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.074194 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.074963 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.075317 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:42:30.072956 24966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.076587 24966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.084734 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.054233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.11.9.124:39185)
0926 07:42:30.054442 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.054450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.066615 (+ 12165us) server_negotiation.cc:407] Connection header received
0926 07:42:30.066845 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.066852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.066941 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.067017 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.068905 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.070136 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.072351 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.073214 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.073317 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.076780 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.076786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.076791 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.076833 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.080756 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.081804 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.081812 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.081819 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.082038 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.082623 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.082630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.082636 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.083192 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.083497 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.083886 (+ 389us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.084209 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:42:30.088585 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.053486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.11.9.126:35101)
0926 07:42:30.053717 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.053725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.065960 (+ 12235us) server_negotiation.cc:407] Connection header received
0926 07:42:30.066251 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.066261 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.066370 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.066466 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.072914 (+ 6448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.073980 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.076120 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.076787 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.076937 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.081517 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.081525 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.081530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.081578 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.084710 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.085801 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.085812 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.085814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.085950 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.086403 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.086408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.086411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.086757 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.087069 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.087789 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.088078 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 07:42:30.095392 24977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:42:30.095516 24977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:42:30.092433 24886 master_runner.cc:186] Error getting master registration for 127.11.9.126:35101: OK, instance_id { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" instance_seqno: 1790408548994337 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:42:30.100746 24886 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:42:30.135373 24805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 4
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408548, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
I20260926 07:42:30.150738 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.135985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42776)
0926 07:42:30.136169 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:30.136218 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:30.136328 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:30.136692 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:30.136696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:30.137102 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:30.137329 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.137339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.138406 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.138412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.139343 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.139354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.140586 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.140591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.140797 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.141611 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:30.141628 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:30.144108 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:30.148184 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.148188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.148191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.148561 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.148980 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.148984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.148986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.149116 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.149825 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:30.149835 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:30.149962 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 07:42:30.150067 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 07:42:30.150414 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 07:42:30.150816 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.136141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.11.9.124:39185)
0926 07:42:30.136330 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.136337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.136355 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:42:30.136446 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.136451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.136541 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.136666 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.137477 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.138239 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.139533 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.140268 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.140388 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.144314 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.144321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.144326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.144363 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.148002 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.148692 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.148698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.148702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.148864 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.149274 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.149279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.149283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.149673 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.149809 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.150222 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.150455 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:42:30.155807 24805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: Bootstrap starting.
I20260926 07:42:30.158993 24805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:30.160122 24805 log.cc:824] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:30.162882 24805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: No bootstrap required, opened a new log
I20260926 07:42:30.170656 24805 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.170929 24805 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:30.171083 24805 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2c56886413432aa5025651527cdeb9, State: Initialized, Role: FOLLOWER
I20260926 07:42:30.171695 24805 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.172646 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.173035 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.173825 24805 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:42:30.185494 24991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:42:30.185626 24991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:42:30.183347 24738 master_runner.cc:186] Error getting master registration for 127.11.9.125:35641: OK, instance_id { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" instance_seqno: 1790408549477785 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:42:30.192337 24738 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:42:30.203745 24879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } attempt: 5
W20260926 07:42:30.203984 24907 tablet.cc:2369] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 07:42:30 dist-test-slave-x492 krb5kdc[24725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408549, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
I20260926 07:42:30.217798 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.204929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.11.9.124:39185)
0926 07:42:30.205128 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:30.205132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:30.205147 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:42:30.205248 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:30.205252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:30.205312 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:30.205426 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:30.206186 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.206912 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.208115 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.208575 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.208713 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.211795 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:30.211800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:30.211803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:30.211832 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:30.214885 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.215586 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.215591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.215595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.215724 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:30.216199 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:30.216207 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:30.216212 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:30.216583 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:30.216747 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:30.217142 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 07:42:30.217428 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:42:30.220084 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:30.204360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39185 (local address 127.0.0.1:42778)
0926 07:42:30.204827 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:30.204849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:30.204965 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:30.205450 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:30.205454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:30.205877 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:30.206032 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.206041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.207063 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.207070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.207950 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:30.207961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:30.208733 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:30.208739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:30.208935 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:30.209595 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:30.209612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:30.211594 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:30.215030 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.215037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.215041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.215444 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.215902 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:30.215907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:30.215910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:30.216083 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:30.216723 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:30.216728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:30.216878 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 07:42:30.217035 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 07:42:30.217357 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260926 07:42:30.224411 24879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: Bootstrap starting.
I20260926 07:42:30.227587 24879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:30.228590 24879 log.cc:824] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:30.231324 24879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: No bootstrap required, opened a new log
W20260926 07:42:30.233785 24759 tablet.cc:2369] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:30.241285 24879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.241623 24879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:30.241766 24879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc55d6043ae47a3aa947e5e469652dc, State: Initialized, Role: FOLLOWER
I20260926 07:42:30.242385 24879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.243341 24994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.243757 24994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.244537 24879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:30.272558 24812 master_runner.cc:263] No leader master found from master 3dc55d6043ae47a3aa947e5e469652dc
I20260926 07:42:30.272722 24812 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:42:30.275250 25005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:42:30.275470 25005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:42:30.350106 24966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:42:30.350452 24966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.352156 24966 leader_election.cc:288] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2c56886413432aa5025651527cdeb9 (127.11.9.126:35101), 3dc55d6043ae47a3aa947e5e469652dc (127.11.9.125:35641)
I20260926 07:42:30.353597 24780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "215b0df3b1fc4a9dbe66886785781919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c56886413432aa5025651527cdeb9" is_pre_election: true
I20260926 07:42:30.354218 24780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 215b0df3b1fc4a9dbe66886785781919 in term 0.
I20260926 07:42:30.355523 24854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "215b0df3b1fc4a9dbe66886785781919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dc55d6043ae47a3aa947e5e469652dc" is_pre_election: true
I20260926 07:42:30.356124 24854 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 215b0df3b1fc4a9dbe66886785781919 in term 0.
I20260926 07:42:30.356891 24903 leader_election.cc:302] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215b0df3b1fc4a9dbe66886785781919, ba2c56886413432aa5025651527cdeb9; no voters:
I20260926 07:42:30.358220 24966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:42:30.358369 24966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:42:30.358474 24966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:30.365293 24966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.368292 24780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "215b0df3b1fc4a9dbe66886785781919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c56886413432aa5025651527cdeb9"
I20260926 07:42:30.368522 24780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:30.372084 24780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 215b0df3b1fc4a9dbe66886785781919 in term 1.
I20260926 07:42:30.372570 24903 leader_election.cc:302] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215b0df3b1fc4a9dbe66886785781919, ba2c56886413432aa5025651527cdeb9; no voters:
I20260926 07:42:30.376098 24854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "215b0df3b1fc4a9dbe66886785781919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dc55d6043ae47a3aa947e5e469652dc"
I20260926 07:42:30.376367 24854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:30.379879 24854 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 215b0df3b1fc4a9dbe66886785781919 in term 1.
I20260926 07:42:30.380744 24966 leader_election.cc:288] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [CANDIDATE]: Term 1 election: Requested vote from peers ba2c56886413432aa5025651527cdeb9 (127.11.9.126:35101), 3dc55d6043ae47a3aa947e5e469652dc (127.11.9.125:35641)
I20260926 07:42:30.381146 24966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:30.381652 24966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [term 1 LEADER]: Becoming Leader. State: Replica: 215b0df3b1fc4a9dbe66886785781919, State: Running, Role: LEADER
I20260926 07:42:30.382464 24966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } }
I20260926 07:42:30.400640 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215b0df3b1fc4a9dbe66886785781919. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.401258 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:30.408349 25009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:30.410811 25009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:30.430536 24854 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [term 1 FOLLOWER]: Refusing update from remote peer 215b0df3b1fc4a9dbe66886785781919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:30.431658 25007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:42:30.439685 24780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [term 1 FOLLOWER]: Refusing update from remote peer 215b0df3b1fc4a9dbe66886785781919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:30.440809 25007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:42:30.466249 24833 tablet.cc:2369] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:30.475315 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215b0df3b1fc4a9dbe66886785781919. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.475674 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.485778 25010 mvcc.cc:205] Tried to move back new op lower bound from 7333513422551728128 to 7333513422379302912. Current Snapshot: MvccSnapshot[applied={T|T < 7333513422551728128}]
I20260926 07:42:30.498185 25012 mvcc.cc:205] Tried to move back new op lower bound from 7333513422551728128 to 7333513422379302912. Current Snapshot: MvccSnapshot[applied={T|T < 7333513422551728128}]
I20260926 07:42:30.498859 24966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.499208 24966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:30.501216 25009 catalog_manager.cc:1347] Generated new cluster ID: 177b6602a66c4a7c8194f41f0604992c
I20260926 07:42:30.501318 25009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:30.504659 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.504894 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.527390 24994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215b0df3b1fc4a9dbe66886785781919. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.531991 24994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.533866 25009 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:42:30.537864 25011 mvcc.cc:205] Tried to move back new op lower bound from 7333513422551728128 to 7333513422379302912. Current Snapshot: MvccSnapshot[applied={T|T < 7333513422551728128}]
I20260926 07:42:30.538204 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.538934 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:30.541098 25009 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:30.546355 24994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "215b0df3b1fc4a9dbe66886785781919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2c56886413432aa5025651527cdeb9" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35101 } } peers { permanent_uuid: "3dc55d6043ae47a3aa947e5e469652dc" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 35641 } } peers { permanent_uuid: "215b0df3b1fc4a9dbe66886785781919" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39185 } } }
I20260926 07:42:30.546594 24994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:42:30.584172 25009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919: Generated new TSK 0
I20260926 07:42:30.585367 25009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:42:31.187738 24991 catalog_manager.cc:1259] Loaded cluster ID: 177b6602a66c4a7c8194f41f0604992c
I20260926 07:42:31.187836 24991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: loading cluster ID for follower catalog manager: success
I20260926 07:42:31.190384 24991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: acquiring CA information for follower catalog manager: success
I20260926 07:42:31.192142 24991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba2c56886413432aa5025651527cdeb9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:42:31.277647 25005 catalog_manager.cc:1259] Loaded cluster ID: 177b6602a66c4a7c8194f41f0604992c
I20260926 07:42:31.277753 25005 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: loading cluster ID for follower catalog manager: success
I20260926 07:42:31.280203 25005 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: acquiring CA information for follower catalog manager: success
I20260926 07:42:31.281968 25005 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3dc55d6043ae47a3aa947e5e469652dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:42:31.697199 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24738
I20260926 07:42:31.698360 24800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:31.808465 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24738
I20260926 07:42:31.829790 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24812
I20260926 07:42:31.830945 24874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:31.887332 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:31.886518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35101 (local address 127.0.0.1:36058)
0926 07:42:31.887049 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:31.887167 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":306,"threads_started":1}
W20260926 07:42:31.887701 24903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.126:35101: connect: Connection refused (error 111)
I20260926 07:42:31.889070 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:31.888746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35101 (local address 127.0.0.1:36074)
0926 07:42:31.888888 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:31.888966 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 07:42:31.889317 24903 consensus_peers.cc:487] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 -> Peer ba2c56886413432aa5025651527cdeb9 (127.11.9.126:35101): Couldn't send request to peer ba2c56886413432aa5025651527cdeb9. Status: Network error: Client connection negotiation failed: client connection to 127.11.9.126:35101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:42:31.939095 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24812
I20260926 07:42:31.960078 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 24886
I20260926 07:42:31.961360 24948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:32.075002 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:31.961051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51332)
0926 07:42:31.961304 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:31.961382 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 07:42:32.076280 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:32.075831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:35641 (local address 127.0.0.1:51340)
0926 07:42:32.076048 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:32.076140 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260926 07:42:32.076529 24905 consensus_peers.cc:487] T 00000000000000000000000000000000 P 215b0df3b1fc4a9dbe66886785781919 -> Peer 3dc55d6043ae47a3aa947e5e469652dc (127.11.9.125:35641): Couldn't send request to peer 3dc55d6043ae47a3aa947e5e469652dc. Status: Network error: Client connection negotiation failed: client connection to 127.11.9.125:35641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:42:32.077325 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 24886
2026-09-26T07:42:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4824 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:42:32 dist-test-slave-x492 krb5kdc[25029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:42:32 dist-test-slave-x492 krb5kdc[25029](info): set up 2 sockets
Sep 26 07:42:32 dist-test-slave-x492 krb5kdc[25029](info): commencing operation
krb5kdc: starting...
W20260926 07:42:33.185741 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:42:33 dist-test-slave-x492 krb5kdc[25029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:42:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:42:33.417013 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42719
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:33.757207 25042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:33.757483 25042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:33.757573 25042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:33.765401 25042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:33.765516 25042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:33.765566 25042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:33.765615 25042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:33.765661 25042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:33.775488 25042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:33.776763 25042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:33.778106 25042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:33.790057 25047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:33.790098 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:33.790128 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:33.790704 25042 server_base.cc:1034] running on GCE node
I20260926 07:42:33.791481 25042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:33.792863 25042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:33.794107 25042 hybrid_clock.cc:648] HybridClock initialized: now 1790408553794068 us; error 59 us; skew 500 ppm
I20260926 07:42:33.794464 25042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:33 dist-test-slave-x492 krb5kdc[25029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408553, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:33.804967 25042 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:33.806620 25042 webserver.cc:460] Webserver started at http://127.11.9.126:37785/ using document root <none> and password file <none>
I20260926 07:42:33.807324 25042 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:33.807472 25042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:33.807798 25042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:33.810912 25042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "f0c8f458535544efa3610cd311623b6a"
format_stamp: "Formatted at 2026-09-26 07:42:33 on dist-test-slave-x492"
server_key: "67c9977cd1a96bdc27de118b0cd229bc"
server_key_iv: "3fc504d4cfde9e5f09378ab34be9a58f"
server_key_version: "encryptionkey@0"
I20260926 07:42:33.811664 25042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "f0c8f458535544efa3610cd311623b6a"
format_stamp: "Formatted at 2026-09-26 07:42:33 on dist-test-slave-x492"
server_key: "67c9977cd1a96bdc27de118b0cd229bc"
server_key_iv: "3fc504d4cfde9e5f09378ab34be9a58f"
server_key_version: "encryptionkey@0"
I20260926 07:42:33.816951 25042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 07:42:33.820569 25057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:33.822060 25042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:42:33.822357 25042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "f0c8f458535544efa3610cd311623b6a"
format_stamp: "Formatted at 2026-09-26 07:42:33 on dist-test-slave-x492"
server_key: "67c9977cd1a96bdc27de118b0cd229bc"
server_key_iv: "3fc504d4cfde9e5f09378ab34be9a58f"
server_key_version: "encryptionkey@0"
I20260926 07:42:33.822572 25042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:33.851094 25042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:33.855485 25042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:33.855824 25042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:33.875670 25042 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42719
I20260926 07:42:33.875706 25108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42719 every 8 connection(s)
I20260926 07:42:33.877208 25042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:33.880151 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25042
I20260926 07:42:33.880399 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:42:33.880996 25109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:33.881119 11301 external_mini_cluster.cc:1428] Setting key 4de3bd56fb8341f60df43ba126f80396
I20260926 07:42:33.893961 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a: Bootstrap starting.
Sep 26 07:42:33 dist-test-slave-x492 krb5kdc[25029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:33.897264 25109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:33.898556 25109 log.cc:824] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:33.901420 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a: No bootstrap required, opened a new log
I20260926 07:42:33.905143 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:33.883081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.11.9.126:42719)
0926 07:42:33.883984 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:33.884001 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:33.884372 (+ 371us) server_negotiation.cc:407] Connection header received
0926 07:42:33.886342 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:33.886363 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:33.886797 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:33.887220 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:33.888797 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:33.890246 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:33.891710 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:33.892449 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:33.892668 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:33.897601 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:33.897609 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:33.897626 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:33.897667 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:33.901009 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:33.901823 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:33.901829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:33.901834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:33.901998 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:33.902367 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:33.902374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:33.902377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:33.902913 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:33.903259 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:33.903742 (+ 483us) server_negotiation.cc:299] Negotiation successful
0926 07:42:33.904172 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":402,"threads_started":1}
I20260926 07:42:33.908953 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:42:33.909584 25109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } }
I20260926 07:42:33.909955 25109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:33.910059 25109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c8f458535544efa3610cd311623b6a, State: Initialized, Role: FOLLOWER
I20260926 07:42:33.910604 25109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } }
I20260926 07:42:33.910861 25109 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:33.910993 25109 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:33.911164 25109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:33.913672 25109 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } }
I20260926 07:42:33.914146 25109 leader_election.cc:302] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0c8f458535544efa3610cd311623b6a; no voters:
I20260926 07:42:33.914752 25109 leader_election.cc:288] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:42:33.914877 25114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:33.915572 25114 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [term 1 LEADER]: Becoming Leader. State: Replica: f0c8f458535544efa3610cd311623b6a, State: Running, Role: LEADER
I20260926 07:42:33.916102 25114 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } }
I20260926 07:42:33.916818 25109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:33.918699 25115 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0c8f458535544efa3610cd311623b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } } }
I20260926 07:42:33.919041 25115 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:33.919768 25116 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0c8f458535544efa3610cd311623b6a. Latest consensus state: current_term: 1 leader_uuid: "f0c8f458535544efa3610cd311623b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c8f458535544efa3610cd311623b6a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42719 } } }
I20260926 07:42:33.920858 25116 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:33.921304 25120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:33.925976 25120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:33.935423 25120 catalog_manager.cc:1347] Generated new cluster ID: 4450dbe58e6f437194c4c36155e32b87
I20260926 07:42:33.935547 25120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:33.949724 25120 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:42:33.950980 25120 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:33.959120 25120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0c8f458535544efa3610cd311623b6a: Generated new TSK 0
I20260926 07:42:33.959954 25120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:42:33.969310 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25042
I20260926 07:42:33.970443 25104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:34.076104 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25042
2026-09-26T07:42:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1993 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:42:34 dist-test-slave-x492 krb5kdc[25137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:42:34 dist-test-slave-x492 krb5kdc[25137](info): set up 2 sockets
Sep 26 07:42:34 dist-test-slave-x492 krb5kdc[25137](info): commencing operation
krb5kdc: starting...
W20260926 07:42:35.165648 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:42:35 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:42:35.386032 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:35.707973 25150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:35.708256 25150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:35.708410 25150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:35.715587 25150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:35.715693 25150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:35.715830 25150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:35.715922 25150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:35.716040 25150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:35.725916 25150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:35.727228 25150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:35.728479 25150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:35.740142 25156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:35.740093 25155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:35.740177 25158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:35.741326 25150 server_base.cc:1034] running on GCE node
I20260926 07:42:35.742094 25150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:35.743444 25150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:35.744684 25150 hybrid_clock.cc:648] HybridClock initialized: now 1790408555744630 us; error 60 us; skew 500 ppm
I20260926 07:42:35.745074 25150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:35 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:35.748284 25150 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:35.749670 25150 webserver.cc:460] Webserver started at http://127.11.9.126:36843/ using document root <none> and password file <none>
I20260926 07:42:35.750314 25150 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:35.750437 25150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:35.750721 25150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:35.753660 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "1ddcc9eef121476d990760744004a994"
format_stamp: "Formatted at 2026-09-26 07:42:35 on dist-test-slave-x492"
server_key: "959640607478e300ca7826cefe871738"
server_key_iv: "f4a0f1ee20c2e817a0972d32597df345"
server_key_version: "encryptionkey@0"
I20260926 07:42:35.754401 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "1ddcc9eef121476d990760744004a994"
format_stamp: "Formatted at 2026-09-26 07:42:35 on dist-test-slave-x492"
server_key: "959640607478e300ca7826cefe871738"
server_key_iv: "f4a0f1ee20c2e817a0972d32597df345"
server_key_version: "encryptionkey@0"
I20260926 07:42:35.759315 25150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:42:35.762548 25165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:35.763801 25150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 07:42:35.763967 25150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "1ddcc9eef121476d990760744004a994"
format_stamp: "Formatted at 2026-09-26 07:42:35 on dist-test-slave-x492"
server_key: "959640607478e300ca7826cefe871738"
server_key_iv: "f4a0f1ee20c2e817a0972d32597df345"
server_key_version: "encryptionkey@0"
I20260926 07:42:35.764147 25150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:35.788962 25150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:35.792809 25150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:35.793092 25150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:35.811126 25150 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:38323
I20260926 07:42:35.811199 25216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:38323 every 8 connection(s)
I20260926 07:42:35.812418 25150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:35.815958 25217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:35.817037 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25150
I20260926 07:42:35.817301 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:42:35.817996 11301 external_mini_cluster.cc:1428] Setting key bfbc6a4a5e52c92ae0520ce4d4ad3d12
I20260926 07:42:35.829845 25217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap starting.
Sep 26 07:42:35 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:35.833515 25217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:35.834692 25217 log.cc:824] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:35.837711 25217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: No bootstrap required, opened a new log
I20260926 07:42:35.840457 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:35.819802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.11.9.126:38323)
0926 07:42:35.820595 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:35.820606 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:35.820645 (+ 39us) server_negotiation.cc:407] Connection header received
0926 07:42:35.822210 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:35.822234 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:35.822586 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:35.822900 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:35.824607 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:35.825910 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:35.827780 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:35.828410 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:35.828532 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:35.833272 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:35.833281 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:35.833294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:35.833332 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:35.836366 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:35.837051 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:35.837066 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:35.837071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:35.837299 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:35.837680 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:35.837685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:35.837689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:35.838298 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:35.838609 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:35.839028 (+ 419us) server_negotiation.cc:299] Negotiation successful
0926 07:42:35.839463 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":390,"threads_started":1}
I20260926 07:42:35.846004 25217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:35.846362 25217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:35.846474 25217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddcc9eef121476d990760744004a994, State: Initialized, Role: FOLLOWER
I20260926 07:42:35.847004 25217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:35.847199 25217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:35.847374 25217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:35.847540 25217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:35.849828 25217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:35.850354 25217 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddcc9eef121476d990760744004a994; no voters:
I20260926 07:42:35.850989 25217 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:42:35.851164 25222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:35.853318 25217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:35.853554 25222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 LEADER]: Becoming Leader. State: Replica: 1ddcc9eef121476d990760744004a994, State: Running, Role: LEADER
I20260926 07:42:35.854192 25222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:35.857515 25224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddcc9eef121476d990760744004a994. Latest consensus state: current_term: 1 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:35.857986 25224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:35.858534 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:35.859018 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:35.861354 25234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:35.865816 25234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:35.874099 25234 catalog_manager.cc:1347] Generated new cluster ID: 580cf088817f4a848958a814a984f299
I20260926 07:42:35.874212 25234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:35.891880 25234 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:42:35.893874 25234 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:35.913452 25234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Generated new TSK 0
I20260926 07:42:35.914330 25234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:42:35.974927 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:36.317413 25243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:36.317713 25243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:36.317869 25243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:36.324864 25243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:36.324970 25243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:36.325225 25243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:42:36.334614 25243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:36.335949 25243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:36.337280 25243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:36.351989 25248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:36.352044 25249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:36.352008 25251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:36.353080 25243 server_base.cc:1034] running on GCE node
I20260926 07:42:36.353720 25243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:36.354705 25243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:36.355952 25243 hybrid_clock.cc:648] HybridClock initialized: now 1790408556355893 us; error 47 us; skew 500 ppm
I20260926 07:42:36.356324 25243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:36 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408556, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:36.359957 25243 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:42:36.361555 25243 webserver.cc:460] Webserver started at http://127.11.9.65:40091/ using document root <none> and password file <none>
I20260926 07:42:36.362215 25243 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:36.362323 25243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:36.362576 25243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:36.365588 25243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "f0378bc17d704eb98dcca5f9b6becf45"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "dc47ccbfec344ef299b421b3e42d18d8"
server_key_iv: "472b195b7bb3a239b4370d67b3b26f97"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.366357 25243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "f0378bc17d704eb98dcca5f9b6becf45"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "dc47ccbfec344ef299b421b3e42d18d8"
server_key_iv: "472b195b7bb3a239b4370d67b3b26f97"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.371888 25243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:42:36.375260 25258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.376506 25243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:42:36.376633 25243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "f0378bc17d704eb98dcca5f9b6becf45"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "dc47ccbfec344ef299b421b3e42d18d8"
server_key_iv: "472b195b7bb3a239b4370d67b3b26f97"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.376791 25243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:36.408803 25243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:36.412766 25243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:36.413004 25243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:36.413936 25243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:36.415251 25243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:36.415321 25243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.415484 25243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:36.415542 25243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.447705 25243 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44877
I20260926 07:42:36.447779 25370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44877 every 8 connection(s)
I20260926 07:42:36.448989 25243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:42:36.452993 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25243
I20260926 07:42:36.453254 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:42:36.454119 11301 external_mini_cluster.cc:1428] Setting key f66de695c61e64d8b39e0b99ce0732f2
Sep 26 07:42:36 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408556, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:36.472802 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:36.451621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:44641 (local address 127.11.9.126:38323)
0926 07:42:36.452063 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:36.452068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:36.453231 (+ 1163us) server_negotiation.cc:407] Connection header received
0926 07:42:36.455802 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:36.455807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:36.455956 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:36.456081 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:36.458212 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.458975 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.461544 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.461942 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.462048 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:36.466756 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:36.466761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:36.466764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:36.466797 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:36.469618 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:36.470546 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:36.470550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:36.470552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:36.470668 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:36.471152 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:36.471155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:36.471157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:36.471402 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:36.471509 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:36.472383 (+ 874us) server_negotiation.cc:299] Negotiation successful
0926 07:42:36.472549 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260926 07:42:36.474381 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:36.452209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:44641)
0926 07:42:36.453035 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:36.453091 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:36.455541 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:36.456359 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:36.456367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:36.457130 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:36.457995 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:36.458024 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.459137 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.459145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:36.461387 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:36.461405 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.462180 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.462188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:36.462533 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:36.463304 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:36.463343 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:36.466535 (+ 3192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:36.469748 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:36.469760 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:36.469780 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:36.470422 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:36.470776 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:36.470784 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:36.470790 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:36.471044 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:36.471530 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:36.471540 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:36.471981 (+ 441us) client_negotiation.cc:769] Sending connection context
0926 07:42:36.472305 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 07:42:36.472878 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":315,"threads_started":1}
I20260926 07:42:36.475989 25371 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:36.476291 25371 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:36.477016 25371 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:42:36.479135 25182 ts_manager.cc:194] Registered new tserver with Master: f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877)
I20260926 07:42:36.480991 25182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:44641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:42:36.508767 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:36.838639 25377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:36.838943 25377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:36.839111 25377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:36.846367 25377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:36.846508 25377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:36.846684 25377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:42:36.856902 25377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:36.858525 25377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:36.859917 25377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:36.873586 25382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:36.873627 25383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:36.873605 25385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:36.874464 25377 server_base.cc:1034] running on GCE node
I20260926 07:42:36.875070 25377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:36.875927 25377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:36.877122 25377 hybrid_clock.cc:648] HybridClock initialized: now 1790408556877079 us; error 34 us; skew 500 ppm
I20260926 07:42:36.877501 25377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:36 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408556, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:36.880861 25377 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:42:36.882300 25377 webserver.cc:460] Webserver started at http://127.11.9.66:38757/ using document root <none> and password file <none>
I20260926 07:42:36.882954 25377 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:36.883095 25377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:36.883390 25377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:36.886327 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "4bbd16437eee45268f97180d1e7c6974"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "de648880cbdbdd961fb6773af696c0e5"
server_key_iv: "f6968775d4db3c1fe5c0526e9c0befc2"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.887094 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "4bbd16437eee45268f97180d1e7c6974"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "de648880cbdbdd961fb6773af696c0e5"
server_key_iv: "f6968775d4db3c1fe5c0526e9c0befc2"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.892009 25377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:42:36.895417 25392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.896693 25377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:42:36.896859 25377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "4bbd16437eee45268f97180d1e7c6974"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "de648880cbdbdd961fb6773af696c0e5"
server_key_iv: "f6968775d4db3c1fe5c0526e9c0befc2"
server_key_version: "encryptionkey@0"
I20260926 07:42:36.897050 25377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:36.919695 25377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:36.923923 25377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:36.924209 25377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:36.925128 25377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:36.926482 25377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:36.926584 25377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.926719 25377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:36.926788 25377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:36.960291 25377 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45503
I20260926 07:42:36.960386 25504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45503 every 8 connection(s)
I20260926 07:42:36.961764 25377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:42:36.969430 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25377
I20260926 07:42:36.969609 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:42:36.970031 11301 external_mini_cluster.cc:1428] Setting key f44ea2aae1f1f7bc359c5d10dcbceacf
Sep 26 07:42:36 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408556, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:36.982949 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:36.964402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:35203 (local address 127.11.9.126:38323)
0926 07:42:36.964590 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:36.964595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:36.965785 (+ 1190us) server_negotiation.cc:407] Connection header received
0926 07:42:36.967579 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:36.967587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:36.967698 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:36.967817 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:36.969270 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.970090 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.971921 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.972346 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.972479 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:36.976535 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:36.976540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:36.976543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:36.976574 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:36.979509 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:36.980314 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:36.980322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:36.980329 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:36.980490 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:36.980896 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:36.980904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:36.980910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:36.981329 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:36.981500 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:36.982220 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 07:42:36.982493 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:42:36.983559 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:36.964887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:35203)
0926 07:42:36.965629 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:36.965670 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:36.967312 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:36.968052 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:36.968060 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:36.968542 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:36.969085 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:36.969101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.970251 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.970254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:36.971776 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:36.971787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:36.972573 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:36.972581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:36.972799 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:36.973523 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:36.973544 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:36.976324 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:36.979657 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:36.979663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:36.979676 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:36.980188 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:36.980593 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:36.980596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:36.980599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:36.980766 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:36.981496 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:36.981502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:36.981820 (+ 318us) client_negotiation.cc:769] Sending connection context
0926 07:42:36.982062 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 07:42:36.982455 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":317,"threads_started":1}
I20260926 07:42:36.984926 25505 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:36.985284 25505 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:36.985994 25505 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
I20260926 07:42:36.987427 25182 ts_manager.cc:194] Registered new tserver with Master: 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:36.988245 25182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:35203
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:42:37.022277 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:37.337819 25510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:37.338138 25510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:37.338304 25510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:37.345628 25510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:37.345790 25510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:37.345963 25510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:42:37.355922 25510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:37.357226 25510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:37.358496 25510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:37.371331 25516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:37.371448 25518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:37.371426 25515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:37.372005 25510 server_base.cc:1034] running on GCE node
I20260926 07:42:37.372531 25510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:37.373431 25510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:37.374616 25510 hybrid_clock.cc:648] HybridClock initialized: now 1790408557374561 us; error 51 us; skew 500 ppm
I20260926 07:42:37.374909 25510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:37 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408557, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:37.377964 25510 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:42:37.379237 25510 webserver.cc:460] Webserver started at http://127.11.9.67:33023/ using document root <none> and password file <none>
I20260926 07:42:37.379848 25510 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:37.380020 25510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:37.380324 25510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:37.383126 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "70027f000f6e4018b5ea3cf1c5140384"
format_stamp: "Formatted at 2026-09-26 07:42:37 on dist-test-slave-x492"
server_key: "8e6d8375b7756713ecec1e0d49a2602f"
server_key_iv: "5e41406a08b9b1d030306238acb0ac7b"
server_key_version: "encryptionkey@0"
I20260926 07:42:37.383827 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "70027f000f6e4018b5ea3cf1c5140384"
format_stamp: "Formatted at 2026-09-26 07:42:37 on dist-test-slave-x492"
server_key: "8e6d8375b7756713ecec1e0d49a2602f"
server_key_iv: "5e41406a08b9b1d030306238acb0ac7b"
server_key_version: "encryptionkey@0"
I20260926 07:42:37.388595 25510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 07:42:37.391966 25525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:37.393307 25510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:42:37.393431 25510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "70027f000f6e4018b5ea3cf1c5140384"
format_stamp: "Formatted at 2026-09-26 07:42:37 on dist-test-slave-x492"
server_key: "8e6d8375b7756713ecec1e0d49a2602f"
server_key_iv: "5e41406a08b9b1d030306238acb0ac7b"
server_key_version: "encryptionkey@0"
I20260926 07:42:37.393590 25510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:37.412853 25510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:37.417100 25510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:37.417411 25510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:37.418314 25510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:37.419519 25510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:37.419589 25510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:37.419746 25510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:37.419792 25510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:37.453037 25510 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38317
I20260926 07:42:37.453086 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38317 every 8 connection(s)
I20260926 07:42:37.454344 25510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:42:37.463114 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25510
I20260926 07:42:37.463299 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:42:37.463774 11301 external_mini_cluster.cc:1428] Setting key a447a95f9d5f4d39c6c6342763884a05
Sep 26 07:42:37 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408557, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:37.473552 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.456920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:44849 (local address 127.11.9.126:38323)
0926 07:42:37.457085 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.457090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.458286 (+ 1196us) server_negotiation.cc:407] Connection header received
0926 07:42:37.460054 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.460060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.460151 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.460260 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:37.461762 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.462545 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.464231 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.464700 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.464831 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.468436 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:37.468445 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:37.468453 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:37.468496 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:37.471017 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.471665 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.471669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.471671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.471795 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.472155 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.472158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.472160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.472409 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:37.472517 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.473095 (+ 578us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.473306 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:42:37.474440 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.457433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:44849)
0926 07:42:37.458135 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.458170 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.459760 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.460490 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.460498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.460988 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:37.461595 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.461613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.462703 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.462707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.464092 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.464105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.464927 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.464950 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.465297 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.465985 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:37.466010 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:37.468211 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:37.471145 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:37.471152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:37.471163 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:37.471550 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:37.471888 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:37.471891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:37.471893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:37.472059 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:37.472505 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:37.472511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:37.472788 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.472995 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.473417 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":285,"threads_started":1}
I20260926 07:42:37.475778 25638 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:37.476037 25638 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:37.476686 25638 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
I20260926 07:42:37.478354 25182 ts_manager.cc:194] Registered new tserver with Master: 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67:38317)
I20260926 07:42:37.479048 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:42:37.479292 25182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:44849
I20260926 07:42:37.484030 25371 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:37.495911 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.483446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.11.9.126:38323)
0926 07:42:37.483590 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.483594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.484089 (+ 495us) server_negotiation.cc:407] Connection header received
0926 07:42:37.484302 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.484306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.484400 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.484499 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:37.486135 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.486839 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.488135 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.488617 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.488743 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.491020 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:37.491025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:37.491028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:37.491055 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:37.493503 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.494173 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.494177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.494179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.494299 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.494611 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.494649 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.494652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.494955 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:37.495136 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.495515 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.495680 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 07:42:37.499828 25182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:42:37.506560 25182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:42:37.535699 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.518726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38317 (local address 127.0.0.1:51984)
0926 07:42:37.523988 (+ 5262us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.524048 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.524271 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.525279 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.525290 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.525332 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:37.525885 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.525898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.529477 (+ 3579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.529483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.532377 (+ 2894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.532402 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.534497 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.534504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.534733 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.534797 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.534917 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.535132 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4946,"mutex_wait_us":73,"thread_start_us":4824,"threads_started":1}
I20260926 07:42:37.536310 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.519705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.11.9.65:44877)
0926 07:42:37.521845 (+ 2140us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.521859 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.527534 (+ 5675us) server_negotiation.cc:407] Connection header received
0926 07:42:37.527741 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.527751 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.528001 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.528189 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:37.528844 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.530678 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.533546 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.534747 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.534879 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.535116 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.535815 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.535980 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1897,"thread_start_us":2241,"threads_started":1}
I20260926 07:42:37.536329 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.519926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.11.9.67:38317)
0926 07:42:37.523258 (+ 3332us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.523267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.524493 (+ 1226us) server_negotiation.cc:407] Connection header received
0926 07:42:37.524639 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.524648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.524971 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.526110 (+ 1139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:37.526444 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.529292 (+ 2848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.532558 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.534271 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.535478 (+ 1207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.535742 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.535813 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.535979 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3150,"thread_start_us":2937,"threads_started":1}
I20260926 07:42:37.536695 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.520011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:45503 (local address 127.0.0.1:34368)
0926 07:42:37.528154 (+ 8143us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.528172 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.528404 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.529158 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.529213 (+ 55us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.529244 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:37.529728 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.529745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.531589 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.531593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.533553 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.533569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.535793 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.535802 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.535986 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.536017 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.536101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.536248 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7973,"mutex_wait_us":48,"thread_start_us":7848,"threads_started":1}
I20260926 07:42:37.537230 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.518992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44877 (local address 127.0.0.1:41984)
0926 07:42:37.527397 (+ 8405us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.527423 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.527584 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.528197 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.528204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.528235 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:37.528622 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.528634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.530869 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.530890 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.533372 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.533389 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.535263 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.535270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.535443 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.535474 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.535564 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.535697 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8221,"mutex_wait_us":2,"thread_start_us":8058,"threads_started":1}
I20260926 07:42:37.537498 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.520438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.11.9.66:45503)
0926 07:42:37.522736 (+ 2298us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.522748 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.528269 (+ 5521us) server_negotiation.cc:407] Connection header received
0926 07:42:37.528629 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.528636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.528850 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.528968 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:37.529943 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.531414 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.533725 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.535266 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.535411 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.535626 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.536940 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.537135 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2074,"thread_start_us":1954,"threads_started":1}
I20260926 07:42:37.540455 25573 tablet_service.cc:1467] Processing CreateTablet for tablet b5f7509b29b34b7d9b435539441cf968 (DEFAULT_TABLE table=test-table [id=87a811ba0a6246c2b0937df777d98739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:37.540784 25306 tablet_service.cc:1467] Processing CreateTablet for tablet b5f7509b29b34b7d9b435539441cf968 (DEFAULT_TABLE table=test-table [id=87a811ba0a6246c2b0937df777d98739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:37.541298 25440 tablet_service.cc:1467] Processing CreateTablet for tablet b5f7509b29b34b7d9b435539441cf968 (DEFAULT_TABLE table=test-table [id=87a811ba0a6246c2b0937df777d98739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:37.542021 25573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f7509b29b34b7d9b435539441cf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:37.542145 25306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f7509b29b34b7d9b435539441cf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:37.542538 25440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f7509b29b34b7d9b435539441cf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:37.560726 25656 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Bootstrap starting.
I20260926 07:42:37.562674 25657 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Bootstrap starting.
I20260926 07:42:37.564404 25656 tablet_bootstrap.cc:654] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:37.566250 25658 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Bootstrap starting.
I20260926 07:42:37.567044 25657 tablet_bootstrap.cc:654] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:37.567466 25656 log.cc:824] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:37.568364 25657 log.cc:824] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:37.570900 25658 tablet_bootstrap.cc:654] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:37.571087 25656 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: No bootstrap required, opened a new log
I20260926 07:42:37.571395 25656 ts_tablet_manager.cc:1397] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260926 07:42:37.572201 25658 log.cc:824] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:37.575220 25658 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: No bootstrap required, opened a new log
I20260926 07:42:37.575519 25658 ts_tablet_manager.cc:1397] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 07:42:37.575553 25657 tablet_bootstrap.cc:492] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: No bootstrap required, opened a new log
I20260926 07:42:37.575793 25657 ts_tablet_manager.cc:1397] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260926 07:42:37.580659 25656 raft_consensus.cc:348] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.581136 25656 raft_consensus.cc:374] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:37.581270 25656 raft_consensus.cc:729] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378bc17d704eb98dcca5f9b6becf45, State: Initialized, Role: FOLLOWER
I20260926 07:42:37.581919 25656 consensus_queue.cc:260] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.583737 25656 ts_tablet_manager.cc:1428] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Time spent starting tablet: real 0.012s user 0.006s sys 0.008s
I20260926 07:42:37.586480 25657 raft_consensus.cc:348] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.586480 25658 raft_consensus.cc:348] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.586925 25658 raft_consensus.cc:374] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:37.586925 25657 raft_consensus.cc:374] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:37.587036 25657 raft_consensus.cc:729] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70027f000f6e4018b5ea3cf1c5140384, State: Initialized, Role: FOLLOWER
I20260926 07:42:37.587028 25658 raft_consensus.cc:729] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbd16437eee45268f97180d1e7c6974, State: Initialized, Role: FOLLOWER
I20260926 07:42:37.587636 25658 consensus_queue.cc:260] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.587637 25657 consensus_queue.cc:260] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.589241 25505 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:37.589346 25638 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:37.589507 25657 ts_tablet_manager.cc:1428] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260926 07:42:37.590983 25658 ts_tablet_manager.cc:1428] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260926 07:42:37.636862 25663 raft_consensus.cc:482] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:42:37.637279 25663 raft_consensus.cc:504] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.639110 25663 leader_election.cc:288] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877), 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:37.645045 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.639490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:57361 (local address 127.11.9.65:44877)
0926 07:42:37.639712 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.639720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.639743 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:42:37.639819 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.639824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.639902 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.640005 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:37.640598 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.641442 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.643241 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.644076 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.644245 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.644355 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.644633 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.644770 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:42:37.645100 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.639282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44877 (local address 127.11.9.67:57361)
0926 07:42:37.639517 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.639552 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.639675 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.640178 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.640189 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.640221 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:37.640445 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.640462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.641585 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.641594 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.643074 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.643086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.644255 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.644261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.644458 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.644484 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.644620 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.644766 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":43}
I20260926 07:42:37.645292 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.640157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:45839 (local address 127.11.9.66:45503)
0926 07:42:37.640361 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.640366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.640393 (+ 27us) server_negotiation.cc:407] Connection header received
0926 07:42:37.640449 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.640452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.640505 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.640603 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:37.641111 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.641850 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.643587 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.644329 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.644505 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.644597 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.644870 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.645000 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:42:37.645623 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.639412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:45503 (local address 127.11.9.67:45839)
0926 07:42:37.640202 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:37.640220 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:37.640335 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:37.640625 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:37.640630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:37.640655 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:37.640953 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.640961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.642009 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.642021 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.643434 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:37.643446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.644546 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.644552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:37.644707 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.644730 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:42:37.644850 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 07:42:37.644979 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":27,"thread_start_us":398,"threads_started":1}
I20260926 07:42:37.646308 25326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f7509b29b34b7d9b435539441cf968" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378bc17d704eb98dcca5f9b6becf45" is_pre_election: true
I20260926 07:42:37.646723 25460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f7509b29b34b7d9b435539441cf968" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbd16437eee45268f97180d1e7c6974" is_pre_election: true
I20260926 07:42:37.646996 25326 raft_consensus.cc:2454] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 0.
I20260926 07:42:37.647303 25460 raft_consensus.cc:2454] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 0.
I20260926 07:42:37.647794 25529 leader_election.cc:302] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70027f000f6e4018b5ea3cf1c5140384, f0378bc17d704eb98dcca5f9b6becf45; no voters:
I20260926 07:42:37.648202 25663 raft_consensus.cc:2781] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:42:37.648375 25663 raft_consensus.cc:482] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:42:37.648484 25663 raft_consensus.cc:3037] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:37.651144 25663 raft_consensus.cc:504] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.651847 25663 leader_election.cc:288] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 election: Requested vote from peers f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877), 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:37.652269 25326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f7509b29b34b7d9b435539441cf968" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378bc17d704eb98dcca5f9b6becf45"
I20260926 07:42:37.652339 25460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f7509b29b34b7d9b435539441cf968" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbd16437eee45268f97180d1e7c6974"
I20260926 07:42:37.652501 25326 raft_consensus.cc:3037] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:37.652565 25460 raft_consensus.cc:3037] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:37.655007 25460 raft_consensus.cc:2454] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 1.
I20260926 07:42:37.655141 25326 raft_consensus.cc:2454] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 1.
I20260926 07:42:37.655439 25527 leader_election.cc:302] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bbd16437eee45268f97180d1e7c6974, 70027f000f6e4018b5ea3cf1c5140384; no voters:
I20260926 07:42:37.655792 25663 raft_consensus.cc:2781] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:37.656234 25663 raft_consensus.cc:686] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 LEADER]: Becoming Leader. State: Replica: 70027f000f6e4018b5ea3cf1c5140384, State: Running, Role: LEADER
I20260926 07:42:37.656708 25663 consensus_queue.cc:237] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:37.661638 25182 catalog_manager.cc:5495] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "70027f000f6e4018b5ea3cf1c5140384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } health_report { overall_health: UNKNOWN } } }
I20260926 07:42:37.691907 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.687004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.11.9.67:38317)
0926 07:42:37.687215 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.687220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.687238 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:42:37.687323 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.687326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.687401 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.687542 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:42:37.687841 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.688536 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.689882 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.690301 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.690399 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.690823 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:42:37.691336 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:42:37.691452 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.691528 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.691607 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 07:42:37.700353 25326 raft_consensus.cc:1261] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Refusing update from remote peer 70027f000f6e4018b5ea3cf1c5140384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:37.700349 25460 raft_consensus.cc:1261] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Refusing update from remote peer 70027f000f6e4018b5ea3cf1c5140384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260926 07:42:37.701433 25372 tablet.cc:2369] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:37.701470 25666 consensus_queue.cc:1035] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:42:37.701946 25663 consensus_queue.cc:1035] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:42:37.705842 25639 tablet.cc:2369] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:37.712037 25669 mvcc.cc:205] Tried to move back new op lower bound from 7333513452329857024 to 7333513452166602752. Current Snapshot: MvccSnapshot[applied={T|T < 7333513452329857024}]
W20260926 07:42:37.713798 25506 tablet.cc:2369] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:37.727408 25180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:37.727797 25180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:37.731716 25180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Sending DeleteTablet for 3 replicas of tablet b5f7509b29b34b7d9b435539441cf968
I20260926 07:42:37.732851 25573 tablet_service.cc:1514] Processing DeleteTablet for tablet b5f7509b29b34b7d9b435539441cf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:37 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51984
I20260926 07:42:37.732863 25306 tablet_service.cc:1514] Processing DeleteTablet for tablet b5f7509b29b34b7d9b435539441cf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:37 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:41984
I20260926 07:42:37.733018 25440 tablet_service.cc:1514] Processing DeleteTablet for tablet b5f7509b29b34b7d9b435539441cf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:37 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:34368
I20260926 07:42:37.734078 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25243
I20260926 07:42:37.742950 25680 tablet_replica.cc:330] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: stopping tablet replica
I20260926 07:42:37.743647 25680 raft_consensus.cc:2229] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:37.744468 25680 raft_consensus.cc:2258] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:37.745800 25679 tablet_replica.cc:330] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: stopping tablet replica
I20260926 07:42:37.746420 25679 raft_consensus.cc:2229] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:42:37.747216 25680 ts_tablet_manager.cc:1905] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:37.747365 25679 raft_consensus.cc:2258] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:37.749317 25681 tablet_replica.cc:330] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: stopping tablet replica
I20260926 07:42:37.749730 25681 raft_consensus.cc:2229] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:37.749975 25679 ts_tablet_manager.cc:1905] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:37.750223 25681 raft_consensus.cc:2258] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:37.753576 25681 ts_tablet_manager.cc:1905] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:37.755460 25680 ts_tablet_manager.cc:1918] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:37.755654 25680 log.cc:1196] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/b5f7509b29b34b7d9b435539441cf968
I20260926 07:42:37.756290 25680 ts_tablet_manager.cc:1939] T b5f7509b29b34b7d9b435539441cf968 P f0378bc17d704eb98dcca5f9b6becf45: Deleting consensus metadata
I20260926 07:42:37.758380 25169 catalog_manager.cc:4841] TS f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877): tablet b5f7509b29b34b7d9b435539441cf968 (table test-table [id=87a811ba0a6246c2b0937df777d98739]) successfully deleted
I20260926 07:42:37.762360 25679 ts_tablet_manager.cc:1918] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:37.762558 25679 log.cc:1196] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/b5f7509b29b34b7d9b435539441cf968
I20260926 07:42:37.763099 25679 ts_tablet_manager.cc:1939] T b5f7509b29b34b7d9b435539441cf968 P 70027f000f6e4018b5ea3cf1c5140384: Deleting consensus metadata
I20260926 07:42:37.763687 25681 ts_tablet_manager.cc:1918] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:37.763854 25681 log.cc:1196] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/b5f7509b29b34b7d9b435539441cf968
I20260926 07:42:37.764369 25681 ts_tablet_manager.cc:1939] T b5f7509b29b34b7d9b435539441cf968 P 4bbd16437eee45268f97180d1e7c6974: Deleting consensus metadata
I20260926 07:42:37.764911 25168 catalog_manager.cc:4841] TS 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67:38317): tablet b5f7509b29b34b7d9b435539441cf968 (table test-table [id=87a811ba0a6246c2b0937df777d98739]) successfully deleted
Sep 26 07:42:37 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:42:37.765929 25167 catalog_manager.cc:4841] TS 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503): tablet b5f7509b29b34b7d9b435539441cf968 (table test-table [id=87a811ba0a6246c2b0937df777d98739]) successfully deleted
I20260926 07:42:37.773386 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.735489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.11.9.65:44877)
0926 07:42:37.735700 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.735707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.743164 (+ 7457us) server_negotiation.cc:407] Connection header received
0926 07:42:37.743381 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.743387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.743491 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.743639 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:37.752762 (+ 9123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.754017 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.756196 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.756864 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.756963 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.767377 (+ 10414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:37.767391 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:37.767416 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:37.767454 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:37.770451 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.771365 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.771371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.771373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.771499 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.771924 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.771927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.771929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.772252 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:37.772498 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.772848 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.773019 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:42:37.774135 25366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:37.863579 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25243
I20260926 07:42:37.889060 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25377
Sep 26 07:42:37 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:42:37.905442 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:37.889993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.11.9.66:45503)
0926 07:42:37.890174 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:37.890178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:37.890196 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:42:37.890407 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:37.890411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:37.890478 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:37.890570 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:37.892418 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.893127 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.894500 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:37.894900 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:37.895037 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:37.899623 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:37.899636 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:37.899651 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:37.899682 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:37.902683 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.903520 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.903526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.903529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.903642 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:37.904032 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:37.904035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:37.904037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:37.904378 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:37.904590 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:37.904947 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 07:42:37.905099 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:42:37.906042 25500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:38.021291 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25377
I20260926 07:42:38.038542 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25510
Sep 26 07:42:38 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:42:38.057029 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:38.039351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.11.9.67:38317)
0926 07:42:38.039548 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:38.039555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:38.039577 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:42:38.039663 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:38.039669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:38.039753 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:38.039877 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:38.041116 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:38.042529 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:38.043797 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:38.044523 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:38.044698 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:38.049455 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:38.049468 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:38.049488 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:38.049549 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:38.053898 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:38.054615 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:38.054626 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:38.054631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:38.054824 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:38.055214 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:38.055219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:38.055222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:38.055765 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:38.056111 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:38.056315 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 07:42:38.056557 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:42:38.058009 25633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:38.177438 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25510
I20260926 07:42:38.192839 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25150
I20260926 07:42:38.193966 25212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:38.303419 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25150
I20260926 07:42:38.324280 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--webserver_interface=127.11.9.126
--webserver_port=36843
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:38.646296 25687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:38.646559 25687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:38.646776 25687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:38.654155 25687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:38.654253 25687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:38.654349 25687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:38.654441 25687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:38.654515 25687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:38.664026 25687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=36843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:38.665315 25687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:38.666604 25687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:38.688292 25693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:38.688251 25694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:38.688313 25696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:38.689111 25687 server_base.cc:1034] running on GCE node
I20260926 07:42:38.689929 25687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:38.691390 25687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:38.692598 25687 hybrid_clock.cc:648] HybridClock initialized: now 1790408558692541 us; error 50 us; skew 500 ppm
I20260926 07:42:38.693023 25687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:38 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408558, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:38.696600 25687 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:38.698360 25687 webserver.cc:460] Webserver started at http://127.11.9.126:36843/ using document root <none> and password file <none>
I20260926 07:42:38.699093 25687 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:38.699249 25687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:38.705034 25687 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 07:42:38.707844 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:38.709128 25687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:42:38.709344 25687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "1ddcc9eef121476d990760744004a994"
format_stamp: "Formatted at 2026-09-26 07:42:35 on dist-test-slave-x492"
server_key: "959640607478e300ca7826cefe871738"
server_key_iv: "f4a0f1ee20c2e817a0972d32597df345"
server_key_version: "encryptionkey@0"
I20260926 07:42:38.710398 25687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:38.727253 25687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:38.731676 25687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:38.731951 25687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:38.750872 25687 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:38323
I20260926 07:42:38.750892 25754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:38323 every 8 connection(s)
I20260926 07:42:38.752457 25687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:38.755117 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25687
I20260926 07:42:38.755940 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:44877
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=40091
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:38.759657 25755 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:42:38.762943 25755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap starting.
I20260926 07:42:38.782390 25755 log.cc:824] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:38.794076 25755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:42:38.794653 25755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap complete.
I20260926 07:42:38.803835 25755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:38.804708 25755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddcc9eef121476d990760744004a994, State: Initialized, Role: FOLLOWER
I20260926 07:42:38.805371 25755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:38.805612 25755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:38.805754 25755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:38.805935 25755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:42:38.809686 25755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:38.810199 25755 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddcc9eef121476d990760744004a994; no voters:
I20260926 07:42:38.810813 25755 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 07:42:38.810950 25759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:42:38.812074 25759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 LEADER]: Becoming Leader. State: Replica: 1ddcc9eef121476d990760744004a994, State: Running, Role: LEADER
I20260926 07:42:38.812716 25759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:38.813038 25755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:38.815248 25761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddcc9eef121476d990760744004a994. Latest consensus state: current_term: 2 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:38.815595 25761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:38.816148 25760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:38.816421 25760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:38.826582 25765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:38.834601 25765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:38.836946 25765 catalog_manager.cc:1259] Loaded cluster ID: 580cf088817f4a848958a814a984f299
I20260926 07:42:38.837306 25765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:38.840585 25765 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 07:42:38.858002 25765 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 07:42:38.869705 25776 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 07:42:38.873996 25765 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:42:38.875757 25765 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Loaded TSK: 0
I20260926 07:42:38.877267 25765 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 07:42:39.100941 25757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:39.101274 25757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:39.101445 25757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:39.108685 25757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:39.108783 25757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:39.108943 25757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:42:39.118865 25757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:44877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=40091
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:39.120365 25757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:39.121678 25757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:39.136251 25783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:39.136332 25785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:39.137190 25782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:39.148424 25757 server_base.cc:1034] running on GCE node
I20260926 07:42:39.149089 25757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:39.150211 25757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:39.151429 25757 hybrid_clock.cc:648] HybridClock initialized: now 1790408559151361 us; error 62 us; skew 500 ppm
I20260926 07:42:39.151778 25757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:39 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408559, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:39.158747 25757 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:42:39.160353 25757 webserver.cc:460] Webserver started at http://127.11.9.65:40091/ using document root <none> and password file <none>
I20260926 07:42:39.161057 25757 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:39.161217 25757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:39.166831 25757 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 07:42:39.169605 25792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.171041 25757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:42:39.171216 25757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "f0378bc17d704eb98dcca5f9b6becf45"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "dc47ccbfec344ef299b421b3e42d18d8"
server_key_iv: "472b195b7bb3a239b4370d67b3b26f97"
server_key_version: "encryptionkey@0"
I20260926 07:42:39.172191 25757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:39.198122 25757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:39.202457 25757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:39.202749 25757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:39.203838 25757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:39.205157 25757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:39.205299 25757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.205425 25757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:39.205510 25757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.238300 25757 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44877
I20260926 07:42:39.238384 25904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44877 every 8 connection(s)
I20260926 07:42:39.239746 25757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:42:39.249688 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25757
I20260926 07:42:39.250293 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:45503
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=38757
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:39 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408559, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:39.262691 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:39.242443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:55485 (local address 127.11.9.126:38323)
0926 07:42:39.243029 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:39.243040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:39.243575 (+ 535us) server_negotiation.cc:407] Connection header received
0926 07:42:39.245648 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:39.245661 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:39.245953 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:39.246212 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:39.247573 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.248810 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.250558 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.251239 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.251369 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:39.255474 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:39.255482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:39.255494 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:39.255530 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:39.258389 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:39.259138 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:39.259144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:39.259149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:39.259300 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:39.259740 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:39.259745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:39.259746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:39.260201 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:39.260412 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:39.261132 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 07:42:39.261539 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":274,"threads_started":1}
I20260926 07:42:39.262691 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:39.242616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:55485)
0926 07:42:39.243412 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:39.243454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:39.245100 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:39.246354 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:39.246361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:39.246883 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:39.247391 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:39.247406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.248965 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.248968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:39.250358 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:39.250370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.251469 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.251478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:39.251810 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:39.252513 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:39.252548 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:39.255216 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:39.258535 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:39.258544 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:39.258565 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:39.258970 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:39.259421 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:39.259426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:39.259430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:39.259624 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:39.260350 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:39.260358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:39.260676 (+ 318us) client_negotiation.cc:769] Sending connection context
0926 07:42:39.260914 (+ 238us) client_negotiation.cc:240] Negotiation successful
0926 07:42:39.261438 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":344,"threads_started":1}
I20260926 07:42:39.265928 25905 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:39.266268 25905 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:39.267200 25905 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
I20260926 07:42:39.269497 25720 ts_manager.cc:194] Registered new tserver with Master: f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877)
I20260926 07:42:39.271015 25720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:55485
W20260926 07:42:39.568902 25909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:39.569242 25909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:39.569389 25909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:39.576741 25909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:39.576865 25909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:39.577039 25909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:42:39.586911 25909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:45503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=38757
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:39.588219 25909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:39.589529 25909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:39.602380 25914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:39.602386 25917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:39.602432 25915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:39.603911 25909 server_base.cc:1034] running on GCE node
I20260926 07:42:39.604468 25909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:39.605350 25909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:39.606539 25909 hybrid_clock.cc:648] HybridClock initialized: now 1790408559606502 us; error 30 us; skew 500 ppm
I20260926 07:42:39.606841 25909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:39 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408559, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:39.609944 25909 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:42:39.611203 25909 webserver.cc:460] Webserver started at http://127.11.9.66:38757/ using document root <none> and password file <none>
I20260926 07:42:39.611831 25909 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:39.611977 25909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:39.617426 25909 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:42:39.619854 25924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.621095 25909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:42:39.621299 25909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "4bbd16437eee45268f97180d1e7c6974"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "de648880cbdbdd961fb6773af696c0e5"
server_key_iv: "f6968775d4db3c1fe5c0526e9c0befc2"
server_key_version: "encryptionkey@0"
I20260926 07:42:39.622277 25909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:39.644037 25909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:39.648169 25909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:39.648402 25909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:39.649348 25909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:39.650600 25909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:39.650686 25909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.650766 25909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:39.650837 25909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:39.682698 25909 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45503
I20260926 07:42:39.682781 26036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45503 every 8 connection(s)
I20260926 07:42:39.684135 25909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:42:39.693512 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 25909
I20260926 07:42:39.694108 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:38317
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=33023
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:42:39 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408559, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:39.705080 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:39.686668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:33879 (local address 127.11.9.126:38323)
0926 07:42:39.686842 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:39.686846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:39.688007 (+ 1161us) server_negotiation.cc:407] Connection header received
0926 07:42:39.689888 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:39.689894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:39.689983 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:39.690090 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:39.691527 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.692214 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.693838 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.694263 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.694376 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:39.698480 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:39.698489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:39.698497 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:39.698545 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:39.701935 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:39.702846 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:39.702851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:39.702853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:39.702971 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:39.703439 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:39.703442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:39.703444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:39.703699 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:39.703804 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:39.704685 (+ 881us) server_negotiation.cc:299] Negotiation successful
0926 07:42:39.704851 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:42:39.706636 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:39.687156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:33879)
0926 07:42:39.687837 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:39.687875 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:39.689628 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:39.690339 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:39.690347 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:39.690829 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:39.691365 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:39.691381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.692376 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.692380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:39.693698 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:39.693709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:39.694486 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:39.694494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:39.694850 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:39.695575 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:39.695613 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:39.698264 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:39.702068 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:39.702078 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:39.702097 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:39.702715 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:39.703079 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:39.703086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:39.703093 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:39.703334 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:39.703823 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:39.703833 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:39.704267 (+ 434us) client_negotiation.cc:769] Sending connection context
0926 07:42:39.704567 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 07:42:39.705135 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":264,"threads_started":1}
I20260926 07:42:39.708070 26037 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:39.708359 26037 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:39.709074 26037 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
I20260926 07:42:39.710459 25720 ts_manager.cc:194] Registered new tserver with Master: 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:39.711354 25720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:33879
W20260926 07:42:40.021391 26040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:40.021703 26040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:40.021863 26040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:40.029129 26040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:40.029340 26040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:40.029515 26040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:42:40.039320 26040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:38317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=33023
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:40.040659 26040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:40.042002 26040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:40.055418 26045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:40.055478 26048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:40.055495 26046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:40.056254 26040 server_base.cc:1034] running on GCE node
I20260926 07:42:40.056845 26040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:40.057849 26040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:40.059041 26040 hybrid_clock.cc:648] HybridClock initialized: now 1790408560059000 us; error 45 us; skew 500 ppm
I20260926 07:42:40.059345 26040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:40 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:40.062450 26040 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:42:40.063890 26040 webserver.cc:460] Webserver started at http://127.11.9.67:33023/ using document root <none> and password file <none>
I20260926 07:42:40.064549 26040 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:40.064697 26040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:40.069967 26040 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:42:40.072723 26055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:40.074113 26040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:42:40.074297 26040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "70027f000f6e4018b5ea3cf1c5140384"
format_stamp: "Formatted at 2026-09-26 07:42:37 on dist-test-slave-x492"
server_key: "8e6d8375b7756713ecec1e0d49a2602f"
server_key_iv: "5e41406a08b9b1d030306238acb0ac7b"
server_key_version: "encryptionkey@0"
I20260926 07:42:40.075251 26040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:40.094396 26040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:40.098748 26040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:40.099047 26040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:40.100024 26040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:40.101403 26040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:40.101534 26040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:40.101671 26040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:40.101763 26040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:40.135272 26040 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38317
I20260926 07:42:40.135358 26167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38317 every 8 connection(s)
I20260926 07:42:40.136817 26040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:42:40.139453 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26040
Sep 26 07:42:40 dist-test-slave-x492 krb5kdc[25137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:40.156216 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.140317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.11.9.126:38323)
0926 07:42:40.140813 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.140822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.141704 (+ 882us) server_negotiation.cc:407] Connection header received
0926 07:42:40.141894 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.141899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.141974 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.142094 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:40.143514 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.144678 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.146634 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.147281 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.147413 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.150624 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:40.150632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:40.150639 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:40.150679 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:40.153510 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.154196 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.154201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.154209 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.154354 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.154728 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.154737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.154739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.155047 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:40.155325 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.155649 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.155880 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":244,"threads_started":1}
I20260926 07:42:40.158866 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.139639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:40737 (local address 127.11.9.126:38323)
0926 07:42:40.139851 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.139858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.141522 (+ 1664us) server_negotiation.cc:407] Connection header received
0926 07:42:40.143588 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.143592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.143649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.143726 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:40.145531 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.146618 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.148444 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.149183 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.149353 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.153218 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:40.153224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:40.153227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:40.153266 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:40.155787 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.156671 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.156679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.156685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.156814 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.157259 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.157266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.157272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.157511 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:40.157673 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.158333 (+ 660us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.158526 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:42:40.159812 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.140176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:40737)
0926 07:42:40.141358 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.141409 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.143296 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.143998 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.144009 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.144686 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:42:40.145373 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.145393 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.146793 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.146799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.148302 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.148314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.149412 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.149416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.149599 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.150244 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:42:40.150270 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:42:40.152983 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:42:40.155924 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:40.155932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:40.155948 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:40.156506 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:40.156924 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:42:40.156929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:42:40.156933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:42:40.157110 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:42:40.157653 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:42:40.157660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:42:40.157991 (+ 331us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.158242 (+ 251us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.158743 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":541,"threads_started":1}
I20260926 07:42:40.160991 26168 heartbeater.cc:347] Connected to a master server at 127.11.9.126:38323
I20260926 07:42:40.161315 26168 heartbeater.cc:464] Registering TS with master...
I20260926 07:42:40.162048 26168 heartbeater.cc:511] Master 127.11.9.126:38323 requested a full tablet report, sending...
I20260926 07:42:40.163338 25720 ts_manager.cc:194] Registered new tserver with Master: 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67:38317)
I20260926 07:42:40.164088 25720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:40737
I20260926 07:42:40.170943 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:42:40.177887 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.171759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.11.9.126:38323)
0926 07:42:40.171902 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.171923 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.172373 (+ 450us) server_negotiation.cc:407] Connection header received
0926 07:42:40.172597 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.172601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.172724 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.172877 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:42:40.173325 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.174033 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.175582 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.176067 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.176198 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.176706 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:42:40.177305 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:42:40.177451 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.177559 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.177639 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 07:42:40.178804 25720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:42:40.186465 25720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:42:40.223377 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.197827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38317 (local address 127.0.0.1:51994)
0926 07:42:40.208386 (+ 10559us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.208443 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.208619 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.215410 (+ 6791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.215414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.215434 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:40.216521 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.216533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.218478 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.218486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.220406 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.220420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.222183 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.222189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.222402 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.222480 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.222627 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.222811 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10325,"mutex_wait_us":2,"thread_start_us":17100,"threads_started":1}
I20260926 07:42:40.223762 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.199408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.11.9.65:44877)
0926 07:42:40.214715 (+ 15307us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.214722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.214745 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:42:40.214824 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.214830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.215026 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.215417 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:40.216087 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.217749 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.221075 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.222267 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.223006 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.223203 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.223270 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.223443 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15156,"thread_start_us":15020,"threads_started":1}
I20260926 07:42:40.223930 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.198762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:45503 (local address 127.0.0.1:34394)
0926 07:42:40.215194 (+ 16432us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.215216 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.215347 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.216056 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.216063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.216087 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:40.216533 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.216541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.217830 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.217837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.220764 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.220778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.223114 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.223121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.223280 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.223323 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.223414 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.223525 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16301,"mutex_wait_us":323,"spinlock_wait_cycles":1280,"thread_start_us":9239,"threads_started":1}
I20260926 07:42:40.224007 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.198155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.11.9.67:38317)
0926 07:42:40.214487 (+ 16332us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.214498 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.214536 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:42:40.214651 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.214659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.214917 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.215123 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:40.216756 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.218348 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.220627 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.221991 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.222123 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.222349 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.223044 (+ 695us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.223203 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16072,"thread_start_us":15942,"threads_started":1}
I20260926 07:42:40.224331 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.198282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44877 (local address 127.0.0.1:42012)
0926 07:42:40.213533 (+ 15251us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.213565 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.213705 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.215271 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.215280 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.215325 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:40.215888 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.215903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.217925 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.217932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.220930 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.220940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.222516 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.222522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.222689 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.222718 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.222772 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.222863 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15103,"mutex_wait_us":225,"spinlock_wait_cycles":3328,"thread_start_us":14895,"threads_started":1}
I20260926 07:42:40.224686 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.199037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.11.9.66:45503)
0926 07:42:40.209263 (+ 10226us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.209276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.215475 (+ 6199us) server_negotiation.cc:407] Connection header received
0926 07:42:40.215644 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.215651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.215856 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.216070 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:40.216682 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.217708 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.220922 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.222516 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.222635 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.222801 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.224261 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.224411 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9978,"thread_start_us":9840,"threads_started":1}
I20260926 07:42:40.227929 25840 tablet_service.cc:1467] Processing CreateTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (DEFAULT_TABLE table=test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:40.227993 26103 tablet_service.cc:1467] Processing CreateTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (DEFAULT_TABLE table=test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:40.227993 25972 tablet_service.cc:1467] Processing CreateTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (DEFAULT_TABLE table=test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:42:40.229137 25840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d5df1b14d4f4491bd3919da4fdfbf18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:40.229270 26103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d5df1b14d4f4491bd3919da4fdfbf18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:40.229281 25972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d5df1b14d4f4491bd3919da4fdfbf18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:40.250556 26184 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Bootstrap starting.
I20260926 07:42:40.250634 26185 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Bootstrap starting.
I20260926 07:42:40.250973 26186 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Bootstrap starting.
I20260926 07:42:40.254547 26186 tablet_bootstrap.cc:654] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:40.255182 26184 tablet_bootstrap.cc:654] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:40.255509 26185 tablet_bootstrap.cc:654] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:40.255834 26186 log.cc:824] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:40.256224 26184 log.cc:824] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:40.256810 26185 log.cc:824] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:40.259203 26186 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: No bootstrap required, opened a new log
I20260926 07:42:40.259339 26184 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: No bootstrap required, opened a new log
I20260926 07:42:40.259517 26186 ts_tablet_manager.cc:1397] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:42:40.259605 26184 ts_tablet_manager.cc:1397] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:42:40.260068 26185 tablet_bootstrap.cc:492] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: No bootstrap required, opened a new log
I20260926 07:42:40.260310 26185 ts_tablet_manager.cc:1397] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260926 07:42:40.267056 26186 raft_consensus.cc:348] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.267412 26186 raft_consensus.cc:374] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:40.267513 26186 raft_consensus.cc:729] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378bc17d704eb98dcca5f9b6becf45, State: Initialized, Role: FOLLOWER
I20260926 07:42:40.268105 26186 consensus_queue.cc:260] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.269213 25905 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:40.269721 26186 ts_tablet_manager.cc:1428] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Time spent starting tablet: real 0.010s user 0.005s sys 0.006s
I20260926 07:42:40.270562 26185 raft_consensus.cc:348] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.270938 26185 raft_consensus.cc:374] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:40.271035 26185 raft_consensus.cc:729] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70027f000f6e4018b5ea3cf1c5140384, State: Initialized, Role: FOLLOWER
I20260926 07:42:40.271596 26185 consensus_queue.cc:260] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.271489 26184 raft_consensus.cc:348] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.271888 26184 raft_consensus.cc:374] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:40.271997 26184 raft_consensus.cc:729] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbd16437eee45268f97180d1e7c6974, State: Initialized, Role: FOLLOWER
I20260926 07:42:40.272646 26168 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:40.273227 26185 ts_tablet_manager.cc:1428] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260926 07:42:40.273699 26184 consensus_queue.cc:260] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.275592 26037 heartbeater.cc:503] Master 127.11.9.126:38323 was elected leader, sending a full tablet report...
I20260926 07:42:40.276136 26184 ts_tablet_manager.cc:1428] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Time spent starting tablet: real 0.016s user 0.004s sys 0.010s
I20260926 07:42:40.336004 26191 raft_consensus.cc:482] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:42:40.336381 26191 raft_consensus.cc:504] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.338346 26191 leader_election.cc:288] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877), 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:40.344430 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.338775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:33003 (local address 127.11.9.65:44877)
0926 07:42:40.339194 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.339202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.339224 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:42:40.339309 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.339315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.339396 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.339493 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:40.339887 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.340803 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.342618 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.343593 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.343693 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.343762 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.344107 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.344191 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260926 07:42:40.344547 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.338583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:44877 (local address 127.11.9.67:33003)
0926 07:42:40.338912 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.338941 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.339117 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.339526 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.339530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.339558 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:40.339756 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.339766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.340934 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.340939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.342462 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.342474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.343747 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.343753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.343944 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.343981 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.344091 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.344223 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260926 07:42:40.345382 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.338765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:50355 (local address 127.11.9.66:45503)
0926 07:42:40.338985 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.338993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.339499 (+ 506us) server_negotiation.cc:407] Connection header received
0926 07:42:40.339667 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.339672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.339749 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.339846 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:42:40.340393 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.341357 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.343227 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.344280 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.344408 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.344495 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.345001 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.345146 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:42:40.345454 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.338651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:45503 (local address 127.11.9.67:50355)
0926 07:42:40.339377 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:40.339394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:42:40.339524 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:42:40.339881 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:42:40.339885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:42:40.339908 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:42:40.340232 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.340243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.341497 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.341500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.343072 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:42:40.343086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.344576 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.344587 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:42:40.344852 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.344875 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:42:40.344946 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 07:42:40.345021 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":4,"thread_start_us":320,"threads_started":1}
I20260926 07:42:40.345676 25860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d5df1b14d4f4491bd3919da4fdfbf18" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378bc17d704eb98dcca5f9b6becf45" is_pre_election: true
I20260926 07:42:40.346184 25860 raft_consensus.cc:2454] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 0.
I20260926 07:42:40.346331 25992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d5df1b14d4f4491bd3919da4fdfbf18" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbd16437eee45268f97180d1e7c6974" is_pre_election: true
I20260926 07:42:40.346812 25992 raft_consensus.cc:2454] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 0.
I20260926 07:42:40.346848 26059 leader_election.cc:302] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70027f000f6e4018b5ea3cf1c5140384, f0378bc17d704eb98dcca5f9b6becf45; no voters:
I20260926 07:42:40.347254 26191 raft_consensus.cc:2781] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:42:40.347402 26191 raft_consensus.cc:482] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:42:40.347514 26191 raft_consensus.cc:3037] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:40.349785 26191 raft_consensus.cc:504] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.350411 26191 leader_election.cc:288] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 election: Requested vote from peers f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877), 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503)
I20260926 07:42:40.350814 25860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d5df1b14d4f4491bd3919da4fdfbf18" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378bc17d704eb98dcca5f9b6becf45"
I20260926 07:42:40.350912 25992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d5df1b14d4f4491bd3919da4fdfbf18" candidate_uuid: "70027f000f6e4018b5ea3cf1c5140384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbd16437eee45268f97180d1e7c6974"
I20260926 07:42:40.350991 25860 raft_consensus.cc:3037] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:40.351133 25992 raft_consensus.cc:3037] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:40.353206 25860 raft_consensus.cc:2454] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 1.
I20260926 07:42:40.353472 25992 raft_consensus.cc:2454] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70027f000f6e4018b5ea3cf1c5140384 in term 1.
I20260926 07:42:40.353562 26059 leader_election.cc:302] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70027f000f6e4018b5ea3cf1c5140384, f0378bc17d704eb98dcca5f9b6becf45; no voters:
I20260926 07:42:40.353838 26191 raft_consensus.cc:2781] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:40.354261 26191 raft_consensus.cc:686] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 LEADER]: Becoming Leader. State: Replica: 70027f000f6e4018b5ea3cf1c5140384, State: Running, Role: LEADER
I20260926 07:42:40.354737 26191 consensus_queue.cc:237] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } }
I20260926 07:42:40.359112 25720 catalog_manager.cc:5495] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67). New cstate: current_term: 1 leader_uuid: "70027f000f6e4018b5ea3cf1c5140384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70027f000f6e4018b5ea3cf1c5140384" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 } health_report { overall_health: UNKNOWN } } }
I20260926 07:42:40.378016 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.372766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.11.9.67:38317)
0926 07:42:40.372976 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.372981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.373014 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:42:40.373302 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.373306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.373378 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.373500 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:42:40.373790 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.374503 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.375888 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.376295 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.376387 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.376878 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:42:40.377447 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:42:40.377637 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.377676 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.377735 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:42:40.385916 25992 raft_consensus.cc:1261] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Refusing update from remote peer 70027f000f6e4018b5ea3cf1c5140384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:40.385926 25860 raft_consensus.cc:1261] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Refusing update from remote peer 70027f000f6e4018b5ea3cf1c5140384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:42:40.386868 26194 consensus_queue.cc:1035] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378bc17d704eb98dcca5f9b6becf45" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:42:40.387185 26191 consensus_queue.cc:1035] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bbd16437eee45268f97180d1e7c6974" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:42:40.388633 26169 tablet.cc:2369] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:42:40.398736 26197 mvcc.cc:205] Tried to move back new op lower bound from 7333513463331344384 to 7333513463216586752. Current Snapshot: MvccSnapshot[applied={T|T < 7333513463331344384}]
I20260926 07:42:40.398710 26196 mvcc.cc:205] Tried to move back new op lower bound from 7333513463331344384 to 7333513463216586752. Current Snapshot: MvccSnapshot[applied={T|T < 7333513463331344384}]
I20260926 07:42:40.413151 25718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:40.413609 25718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:42:40.417388 25718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Sending DeleteTablet for 3 replicas of tablet 9d5df1b14d4f4491bd3919da4fdfbf18
I20260926 07:42:40.418442 26103 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:40 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51994
I20260926 07:42:40.418622 25840 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:40 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:42012
I20260926 07:42:40.418988 25972 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d5df1b14d4f4491bd3919da4fdfbf18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:42:40 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:34394
I20260926 07:42:40.419936 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25757
I20260926 07:42:40.431026 26206 tablet_replica.cc:330] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: stopping tablet replica
I20260926 07:42:40.431805 26206 raft_consensus.cc:2229] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:42:40.433020 26206 raft_consensus.cc:2258] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:40.435415 26206 ts_tablet_manager.cc:1905] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:40.436600 26207 tablet_replica.cc:330] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: stopping tablet replica
I20260926 07:42:40.437077 26207 raft_consensus.cc:2229] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:40.437604 26207 raft_consensus.cc:2258] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:40.440610 26207 ts_tablet_manager.cc:1905] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:40.443643 26206 ts_tablet_manager.cc:1918] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:40.443862 26206 log.cc:1196] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/9d5df1b14d4f4491bd3919da4fdfbf18
I20260926 07:42:40.444515 26206 ts_tablet_manager.cc:1939] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 70027f000f6e4018b5ea3cf1c5140384: Deleting consensus metadata
I20260926 07:42:40.445325 26208 tablet_replica.cc:330] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: stopping tablet replica
I20260926 07:42:40.445744 26208 raft_consensus.cc:2229] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:42:40.446250 26208 raft_consensus.cc:2258] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:42:40.446559 25706 catalog_manager.cc:4841] TS 70027f000f6e4018b5ea3cf1c5140384 (127.11.9.67:38317): tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (table test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]) successfully deleted
I20260926 07:42:40.447278 26207 ts_tablet_manager.cc:1918] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:40.447451 26207 log.cc:1196] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/9d5df1b14d4f4491bd3919da4fdfbf18
I20260926 07:42:40.447995 26207 ts_tablet_manager.cc:1939] T 9d5df1b14d4f4491bd3919da4fdfbf18 P f0378bc17d704eb98dcca5f9b6becf45: Deleting consensus metadata
I20260926 07:42:40.448848 26208 ts_tablet_manager.cc:1905] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:42:40.449937 25707 catalog_manager.cc:4841] TS f0378bc17d704eb98dcca5f9b6becf45 (127.11.9.65:44877): tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (table test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]) successfully deleted
I20260926 07:42:40.450033 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.420874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.11.9.65:44877)
0926 07:42:40.421097 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.421105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.421131 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:42:40.421345 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.421353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.421453 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.421579 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:40.423313 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.424785 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.426787 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.427684 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.427852 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.431400 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:40.431417 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:40.431436 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:40.431480 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:40.436297 (+ 4817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.437963 (+ 1666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.437973 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.437976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.438137 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.438572 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.438577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.438580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.439067 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:40.439353 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.449402 (+ 10049us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.449657 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:42:40.451095 25900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:40.454699 26208 ts_tablet_manager.cc:1918] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:42:40.454908 26208 log.cc:1196] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/9d5df1b14d4f4491bd3919da4fdfbf18
I20260926 07:42:40.455792 26208 ts_tablet_manager.cc:1939] T 9d5df1b14d4f4491bd3919da4fdfbf18 P 4bbd16437eee45268f97180d1e7c6974: Deleting consensus metadata
I20260926 07:42:40.458099 25705 catalog_manager.cc:4841] TS 4bbd16437eee45268f97180d1e7c6974 (127.11.9.66:45503): tablet 9d5df1b14d4f4491bd3919da4fdfbf18 (table test-table [id=e8b4fd80d4bd467ab0ccbe82270198b9]) successfully deleted
I20260926 07:42:40.542315 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25757
I20260926 07:42:40.560065 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25909
I20260926 07:42:40.581342 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.561139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.11.9.66:45503)
0926 07:42:40.561357 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.561361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.561378 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:42:40.561431 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.561435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.561493 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.561569 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:40.563372 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.564061 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.566254 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.566910 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.567020 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.575433 (+ 8413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:40.575448 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:40.575464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:40.575511 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:40.578407 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.579147 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.579160 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.579163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.579298 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.579666 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.579672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.579675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.580083 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:40.580385 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.580673 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.580906 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 07:42:40.582077 26032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:40.668983 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25909
I20260926 07:42:40.686185 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26040
I20260926 07:42:40.701923 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:40.687048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.11.9.67:38317)
0926 07:42:40.687243 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:40.687251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:40.687277 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:42:40.687348 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:40.687354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:40.687447 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:40.687553 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:40.688783 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.690095 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.691356 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:40.692067 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:40.692202 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:40.694990 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:40.695004 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:40.695036 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:40.695080 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:40.699003 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.699711 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.699727 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.699732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.699932 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:40.700289 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:40.700293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:40.700297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:40.700710 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:40.701084 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:40.701267 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 07:42:40.701491 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:42:40.702757 26163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:40.825672 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26040
I20260926 07:42:40.846022 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 25687
I20260926 07:42:40.847194 25750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:40.962441 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 25687
I20260926 07:42:40.984339 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--webserver_interface=127.11.9.126
--webserver_port=36843
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:41.309458 26213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:41.309808 26213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:41.309952 26213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:41.317361 26213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:41.317510 26213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:41.317621 26213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:41.317720 26213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:41.317800 26213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:41.327428 26213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=36843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:41.328733 26213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:41.330024 26213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:41.341596 26219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:41.341678 26218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:41.341701 26221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:41.342243 26213 server_base.cc:1034] running on GCE node
I20260926 07:42:41.342952 26213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:41.344359 26213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:41.345569 26213 hybrid_clock.cc:648] HybridClock initialized: now 1790408561345517 us; error 42 us; skew 500 ppm
I20260926 07:42:41.345928 26213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:41 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:41.348812 26213 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:41.350332 26213 webserver.cc:460] Webserver started at http://127.11.9.126:36843/ using document root <none> and password file <none>
I20260926 07:42:41.351008 26213 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:41.351148 26213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:41.356472 26213 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:42:41.359267 26228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:41.360460 26213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:42:41.360631 26213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "1ddcc9eef121476d990760744004a994"
format_stamp: "Formatted at 2026-09-26 07:42:35 on dist-test-slave-x492"
server_key: "959640607478e300ca7826cefe871738"
server_key_iv: "f4a0f1ee20c2e817a0972d32597df345"
server_key_version: "encryptionkey@0"
I20260926 07:42:41.361605 26213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:41.374169 26213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:41.377847 26213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:41.378093 26213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:41.396060 26213 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:38323
I20260926 07:42:41.396107 26279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:38323 every 8 connection(s)
I20260926 07:42:41.397552 26213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:41.404560 26280 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:42:41.405442 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26213
I20260926 07:42:41.406459 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:44877
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=40091
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:41.408456 26280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap starting.
I20260926 07:42:41.436760 26280 log.cc:824] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:41.457540 26280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:42:41.458179 26280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994: Bootstrap complete.
I20260926 07:42:41.472998 26280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:41.474666 26280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddcc9eef121476d990760744004a994, State: Initialized, Role: FOLLOWER
I20260926 07:42:41.475502 26280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:41.475792 26280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:41.475914 26280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:41.476095 26280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 2 FOLLOWER]: Advancing to term 3
I20260926 07:42:41.481794 26280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:41.482370 26280 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddcc9eef121476d990760744004a994; no voters:
I20260926 07:42:41.483012 26280 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 07:42:41.483155 26284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 07:42:41.484329 26284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [term 3 LEADER]: Becoming Leader. State: Replica: 1ddcc9eef121476d990760744004a994, State: Running, Role: LEADER
I20260926 07:42:41.484844 26284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } }
I20260926 07:42:41.485144 26280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:41.487185 26286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddcc9eef121476d990760744004a994. Latest consensus state: current_term: 3 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:41.487596 26286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:41.488198 26285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1ddcc9eef121476d990760744004a994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddcc9eef121476d990760744004a994" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 38323 } } }
I20260926 07:42:41.488519 26285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddcc9eef121476d990760744004a994 [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:41.498165 26290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:41.507426 26290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:41.508891 26290 catalog_manager.cc:1259] Loaded cluster ID: 580cf088817f4a848958a814a984f299
I20260926 07:42:41.508985 26290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 07:42:41.511104 26290 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8dc3b8d9ad google::LogMessage::Fail() at ??:0
@ 0x7f8dc3b9088a google::LogMessage::SendToLog() at ??:0
@ 0x7f8dc3b8d4de google::LogMessage::Flush() at ??:0
@ 0x7f8dc3b8ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8dd95c0512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8dd95bd976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8dd9645610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8dd9645439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8dc67d9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8dc67dc91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8dc67dc799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8dc67b7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8dc9a916db start_thread at ??:0
@ 0x7f8dc2a2761f clone at ??:0
W20260926 07:42:41.792868 26282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:41.793212 26282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:41.793376 26282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:41.800433 26282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:41.800572 26282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:41.800745 26282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:42:41.810256 26282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:44877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=40091
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:41.811480 26282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:41.812762 26282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:41.825423 26325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:41.825527 26322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:41.825512 26323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:41.826260 26282 server_base.cc:1034] running on GCE node
I20260926 07:42:41.826794 26282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:41.827637 26282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:41.828824 26282 hybrid_clock.cc:648] HybridClock initialized: now 1790408561828792 us; error 24 us; skew 500 ppm
I20260926 07:42:41.829092 26282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:41 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:41.832192 26282 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:42:41.833513 26282 webserver.cc:460] Webserver started at http://127.11.9.65:40091/ using document root <none> and password file <none>
I20260926 07:42:41.834134 26282 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:41.834231 26282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:41.839221 26282 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:42:41.841966 26332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:41.843266 26282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:42:41.843436 26282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "f0378bc17d704eb98dcca5f9b6becf45"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "dc47ccbfec344ef299b421b3e42d18d8"
server_key_iv: "472b195b7bb3a239b4370d67b3b26f97"
server_key_version: "encryptionkey@0"
I20260926 07:42:41.844420 26282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:41.867592 26282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:41.871963 26282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:41.872231 26282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:41.873224 26282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:41.874532 26282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:41.874651 26282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:41.874773 26282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:41.874863 26282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:41.908223 26282 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:44877
I20260926 07:42:41.908336 26444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:44877 every 8 connection(s)
I20260926 07:42:41.909798 26282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:42:41.912258 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26282
I20260926 07:42:41.912843 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:45503
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=38757
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:41.914906 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:41.912808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:56917)
0926 07:42:41.913742 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:41.913988 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":405,"threads_started":1}
W20260926 07:42:41.915604 26445 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:38323 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:38323: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
I20260926 07:42:41.916464 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:41.916152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:48411)
0926 07:42:41.916303 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:41.916366 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 07:42:41.917379 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:41.917045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:38535)
0926 07:42:41.917202 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:41.917278 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 07:42:41.917634 26445 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:42:42.244139 26448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:42.244431 26448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:42.244666 26448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:42.252483 26448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:42.252583 26448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:42.252753 26448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:42:42.262290 26448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:45503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=38757
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:42.263517 26448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:42.264792 26448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:42.277876 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:42.277917 26454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:42.278019 26456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:42.278657 26448 server_base.cc:1034] running on GCE node
I20260926 07:42:42.279250 26448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:42.280220 26448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:42.281430 26448 hybrid_clock.cc:648] HybridClock initialized: now 1790408562281381 us; error 56 us; skew 500 ppm
I20260926 07:42:42.281726 26448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:42 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408562, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:42.284888 26448 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:42:42.286242 26448 webserver.cc:460] Webserver started at http://127.11.9.66:38757/ using document root <none> and password file <none>
I20260926 07:42:42.286850 26448 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:42.286949 26448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:42.291870 26448 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 07:42:42.296586 26463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.297926 26448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 07:42:42.298097 26448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "4bbd16437eee45268f97180d1e7c6974"
format_stamp: "Formatted at 2026-09-26 07:42:36 on dist-test-slave-x492"
server_key: "de648880cbdbdd961fb6773af696c0e5"
server_key_iv: "f6968775d4db3c1fe5c0526e9c0befc2"
server_key_version: "encryptionkey@0"
I20260926 07:42:42.299057 26448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:42.321314 26448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:42.325619 26448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:42.325886 26448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:42.326855 26448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:42.328187 26448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:42.328293 26448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.328419 26448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:42.328517 26448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.361037 26448 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:45503
I20260926 07:42:42.361133 26575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:45503 every 8 connection(s)
I20260926 07:42:42.362532 26448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:42:42.364661 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26448
I20260926 07:42:42.365305 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:38317
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:38323
--webserver_port=33023
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:42:42.367751 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.365654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:34185)
0926 07:42:42.366489 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.366754 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":340,"threads_started":1}
W20260926 07:42:42.368470 26576 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:38323 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:38323: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
I20260926 07:42:42.369378 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.369031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:58405)
0926 07:42:42.369219 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.369282 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 07:42:42.370205 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.369899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:48411)
0926 07:42:42.370029 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.370102 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 07:42:42.370414 26576 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:42:42.688907 26579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:42.689229 26579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:42.689399 26579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:42.696406 26579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:42.696530 26579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:42.696717 26579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:42:42.706300 26579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:38317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=33023
--tserver_master_addrs=127.11.9.126:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:42.707499 26579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:42.708788 26579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:42.722164 26584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:42.722123 26585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:42.722255 26587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:42.722834 26579 server_base.cc:1034] running on GCE node
I20260926 07:42:42.723366 26579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:42.724206 26579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:42.725391 26579 hybrid_clock.cc:648] HybridClock initialized: now 1790408562725340 us; error 47 us; skew 500 ppm
I20260926 07:42:42.725703 26579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:42 dist-test-slave-x492 krb5kdc[25137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408562, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:42.728833 26579 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:42:42.730134 26579 webserver.cc:460] Webserver started at http://127.11.9.67:33023/ using document root <none> and password file <none>
I20260926 07:42:42.730763 26579 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:42.730887 26579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:42.735993 26579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 07:42:42.738592 26594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.739876 26579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:42:42.740046 26579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "70027f000f6e4018b5ea3cf1c5140384"
format_stamp: "Formatted at 2026-09-26 07:42:37 on dist-test-slave-x492"
server_key: "8e6d8375b7756713ecec1e0d49a2602f"
server_key_iv: "5e41406a08b9b1d030306238acb0ac7b"
server_key_version: "encryptionkey@0"
I20260926 07:42:42.740976 26579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:42.767867 26579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:42.771806 26579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:42.772087 26579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:42.773109 26579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:42:42.774447 26579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:42:42.774566 26579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.774693 26579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:42:42.774778 26579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:42.807287 26579 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38317
I20260926 07:42:42.807369 26706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38317 every 8 connection(s)
I20260926 07:42:42.808713 26579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:42:42.813643 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.811725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:42259)
0926 07:42:42.812452 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.812648 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":290,"threads_started":1}
W20260926 07:42:42.814357 26707 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:38323 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:38323: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
I20260926 07:42:42.815109 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.814808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:41569)
0926 07:42:42.814937 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.815016 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 07:42:42.815907 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.815587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:59319)
0926 07:42:42.815733 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.815808 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 07:42:42.816147 26707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 07:42:42.817625 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26579
I20260926 07:42:42.918939 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:42.918182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:51665)
0926 07:42:42.918627 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:42.918716 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":221,"threads_started":1}
I20260926 07:42:43.371658 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:43.370888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:45599)
0926 07:42:43.371337 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:43.371426 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260926 07:42:43.817378 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:43.816587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:60399)
0926 07:42:43.817004 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:43.817092 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":205,"threads_started":1}
I20260926 07:42:43.920815 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:43.919917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:36437)
0926 07:42:43.920539 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:43.920644 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":277,"threads_started":1}
I20260926 07:42:44.373327 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:44.372530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:54057)
0926 07:42:44.373021 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:44.373098 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
I20260926 07:42:44.819015 26716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:44.818316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:50617)
0926 07:42:44.818784 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:44.818859 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
I20260926 07:42:44.922433 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:44.921764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:44721)
0926 07:42:44.922166 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:44.922263 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":222,"threads_started":1}
I20260926 07:42:45.374814 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:45.374129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:40969)
0926 07:42:45.374570 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:45.374644 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260926 07:42:45.820521 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:45.819826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:56317)
0926 07:42:45.820281 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:45.820357 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":254,"threads_started":1}
I20260926 07:42:45.923971 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:45.923259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:40231)
0926 07:42:45.923684 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:45.923763 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":217,"threads_started":1}
I20260926 07:42:46.376343 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:46.375644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:46127)
0926 07:42:46.376075 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:46.376154 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":211,"threads_started":1}
I20260926 07:42:46.822230 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:46.821423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:56409)
0926 07:42:46.821907 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:46.821984 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":270,"threads_started":1}
I20260926 07:42:46.925647 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:46.924897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:45005)
0926 07:42:46.925400 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:46.925479 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":222,"threads_started":1}
I20260926 07:42:47.378054 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:47.377360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:43249)
0926 07:42:47.377795 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:47.377875 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":220,"threads_started":1}
I20260926 07:42:47.823959 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:47.823066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:44435)
0926 07:42:47.823653 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:47.823752 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":299,"threads_started":1}
I20260926 07:42:47.927145 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:47.926460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:56025)
0926 07:42:47.926896 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:47.926982 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":224,"threads_started":1}
I20260926 07:42:48.379616 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:48.378930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:34919)
0926 07:42:48.379360 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:48.379448 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260926 07:42:48.825693 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:48.824785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:44513)
0926 07:42:48.825381 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:48.825496 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":316,"threads_started":1}
I20260926 07:42:48.928764 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:48.928017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:57541)
0926 07:42:48.928475 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:48.928554 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":186,"threads_started":1}
I20260926 07:42:49.381335 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:49.380471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:44753)
0926 07:42:49.380998 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:49.381100 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":201,"threads_started":1}
I20260926 07:42:49.827371 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:49.826549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:55377)
0926 07:42:49.827039 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:49.827116 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":223,"threads_started":1}
I20260926 07:42:49.930405 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:49.929734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:36921)
0926 07:42:49.930176 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:49.930254 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":236,"threads_started":1}
I20260926 07:42:50.383074 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:50.382321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:58107)
0926 07:42:50.382826 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:50.382910 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":262,"threads_started":1}
I20260926 07:42:50.828938 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:50.828229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:39479)
0926 07:42:50.828693 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:50.828778 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":247,"threads_started":1}
I20260926 07:42:50.932085 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:50.931239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:55047)
0926 07:42:50.931707 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:50.931854 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":230,"threads_started":1}
I20260926 07:42:51.384644 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:51.383926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:60357)
0926 07:42:51.384396 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:51.384475 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260926 07:42:51.830593 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:51.829865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:40435)
0926 07:42:51.830347 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:51.830426 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":205,"threads_started":1}
I20260926 07:42:51.933830 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:51.933078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:55063)
0926 07:42:51.933586 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:51.933667 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":311,"threads_started":1}
I20260926 07:42:52.386376 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:52.385610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:34931)
0926 07:42:52.386132 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:52.386210 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":315,"threads_started":1}
I20260926 07:42:52.832212 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:52.831495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:55063)
0926 07:42:52.831968 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:52.832048 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":276,"threads_started":1}
I20260926 07:42:52.935434 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:52.934732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:34345)
0926 07:42:52.935148 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:52.935248 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":200,"threads_started":1}
I20260926 07:42:53.388032 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:53.387226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:59105)
0926 07:42:53.387710 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:53.387835 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":235,"threads_started":1}
I20260926 07:42:53.833865 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:53.833131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:33707)
0926 07:42:53.833609 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:53.833698 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":202,"threads_started":1}
I20260926 07:42:53.937129 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:53.936434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:54733)
0926 07:42:53.936880 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:53.936961 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":218,"threads_started":1}
I20260926 07:42:54.389787 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:54.389048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:37265)
0926 07:42:54.389536 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:54.389620 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":273,"threads_started":1}
I20260926 07:42:54.835377 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:54.834735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:54699)
0926 07:42:54.835158 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:54.835233 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":231,"threads_started":1}
I20260926 07:42:54.938944 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:54.938129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:37453)
0926 07:42:54.938645 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:54.938744 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":248,"threads_started":1}
I20260926 07:42:55.391422 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:55.390674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:42747)
0926 07:42:55.391125 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:55.391234 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":216,"threads_started":1}
I20260926 07:42:55.837005 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:55.836286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:41291)
0926 07:42:55.836730 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:55.836818 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":213,"threads_started":1}
I20260926 07:42:55.940508 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:55.939823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:45283)
0926 07:42:55.940252 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:55.940333 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":216,"threads_started":1}
I20260926 07:42:56.392931 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:56.392266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:49043)
0926 07:42:56.392680 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:56.392754 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":201,"threads_started":1}
I20260926 07:42:56.838567 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:56.837910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:51005)
0926 07:42:56.838314 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:56.838394 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":207,"threads_started":1}
I20260926 07:42:56.942557 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:56.941714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.65:46107)
0926 07:42:56.942240 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:56.942360 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":251,"threads_started":1}
I20260926 07:42:57.394709 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:57.393985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.66:36051)
0926 07:42:57.394413 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:57.394522 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":220,"threads_started":1}
I20260926 07:42:57.820791 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26282
I20260926 07:42:57.838707 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:57.821973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.11.9.65:44877)
0926 07:42:57.822578 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:57.822596 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:57.822642 (+ 46us) server_negotiation.cc:407] Connection header received
0926 07:42:57.824826 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:57.824836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:57.825333 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:57.825633 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:57.827003 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:57.828218 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:57.829500 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:57.830149 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:57.830284 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:57.832927 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:57.832936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:57.832950 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:57.832988 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:57.835695 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:57.836419 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:57.836424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:57.836430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:57.836585 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:57.836983 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:57.836986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:57.836988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:57.837533 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:57.837755 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:57.838105 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 07:42:57.838432 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":291,"threads_started":1}
I20260926 07:42:57.840351 26440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:57.840420 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:57.839696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:38323 (local address 127.11.9.67:56351)
0926 07:42:57.840151 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 07:42:57.840260 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":256,"threads_started":1}
I20260926 07:42:57.923985 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26282
I20260926 07:42:57.940733 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26448
I20260926 07:42:57.958783 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:57.941802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.11.9.66:45503)
0926 07:42:57.942364 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:57.942378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:57.942415 (+ 37us) server_negotiation.cc:407] Connection header received
0926 07:42:57.943949 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:57.943958 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:57.944401 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:57.944791 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:57.946252 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:57.947433 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:57.948680 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:57.949441 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:57.949652 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:57.952271 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:57.952279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:57.952295 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:57.952335 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:57.955281 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:57.956028 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:57.956034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:57.956041 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:57.956208 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:57.956597 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:57.956602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:57.956604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:57.957154 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:57.957621 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:57.958081 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 07:42:57.958484 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":296,"threads_started":1}
I20260926 07:42:57.960211 26571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:58.045851 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26448
I20260926 07:42:58.064023 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26579
I20260926 07:42:58.081949 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:58.064981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.11.9.67:38317)
0926 07:42:58.065624 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:58.065638 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:58.065681 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:42:58.067286 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:58.067294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:58.067731 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:58.068016 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:58.069379 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:58.070760 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:58.072081 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:58.072805 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:58.072955 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:58.075511 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:58.075519 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:58.075535 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:58.075577 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:58.078674 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:58.079285 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:58.079292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:58.079298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:58.079475 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:58.079852 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:58.079856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:58.079858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:58.080389 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:58.080651 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:58.080981 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 07:42:58.081618 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":338,"threads_started":1}
I20260926 07:42:58.083312 26702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:42:58.165462 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26579
2026-09-26T07:42:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24093 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:42:58 dist-test-slave-x492 krb5kdc[26764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:42:58 dist-test-slave-x492 krb5kdc[26764](info): set up 2 sockets
Sep 26 07:42:58 dist-test-slave-x492 krb5kdc[26764](info): commencing operation
krb5kdc: starting...
W20260926 07:42:59.257776 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:42:59 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:42:59.485986 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:46499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:42:59.810528 26777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:42:59.810863 26777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:42:59.811044 26777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:42:59.818248 26777 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 07:42:59.818387 26777 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 07:42:59.818459 26777 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 07:42:59.818538 26777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:42:59.818645 26777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:42:59.818753 26777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:42:59.818843 26777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:42:59.818922 26777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:42:59.828629 26777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:46499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:42:59.830073 26777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:42:59.831456 26777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:42:59.843027 26783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:59.843111 26782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:42:59.843119 26785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:42:59.844141 26777 server_base.cc:1034] running on GCE node
I20260926 07:42:59.844851 26777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:42:59.846194 26777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:42:59.847436 26777 hybrid_clock.cc:648] HybridClock initialized: now 1790408579847375 us; error 33 us; skew 500 ppm
I20260926 07:42:59.847756 26777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:42:59 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:42:59.850723 26777 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:42:59.852173 26777 webserver.cc:460] Webserver started at http://127.11.9.126:33745/ using document root <none> and password file <none>
I20260926 07:42:59.852855 26777 fs_manager.cc:359] Metadata directory not provided
I20260926 07:42:59.852981 26777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:42:59.853369 26777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:42:59.856163 26777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "f73e5bea656743159c18152df1e340ee"
format_stamp: "Formatted at 2026-09-26 07:42:59 on dist-test-slave-x492"
server_key: "a049d55d3a18ad022ecceb02cee12734"
server_key_iv: "b396b8bd3cefb5b81042d068b83b048e"
server_key_version: "encryptionkey@0"
I20260926 07:42:59.856928 26777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "f73e5bea656743159c18152df1e340ee"
format_stamp: "Formatted at 2026-09-26 07:42:59 on dist-test-slave-x492"
server_key: "a049d55d3a18ad022ecceb02cee12734"
server_key_iv: "b396b8bd3cefb5b81042d068b83b048e"
server_key_version: "encryptionkey@0"
I20260926 07:42:59.861810 26777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 07:42:59.865041 26792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:42:59.866318 26777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:42:59.866489 26777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "f73e5bea656743159c18152df1e340ee"
format_stamp: "Formatted at 2026-09-26 07:42:59 on dist-test-slave-x492"
server_key: "a049d55d3a18ad022ecceb02cee12734"
server_key_iv: "b396b8bd3cefb5b81042d068b83b048e"
server_key_version: "encryptionkey@0"
I20260926 07:42:59.866693 26777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:42:59.883113 26777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:42:59.886839 26777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:42:59.887113 26777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:42:59.906270 26777 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:46499
I20260926 07:42:59.906306 26843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:46499 every 8 connection(s)
I20260926 07:42:59.907603 26777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:42:59.911288 26844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:42:59.917824 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26777
I20260926 07:42:59.918056 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:42:59.918911 11301 external_mini_cluster.cc:1428] Setting key 8a63ff771032872804e6c128e4cb0d1e
I20260926 07:42:59.922601 26844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap starting.
I20260926 07:42:59.926175 26844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Neither blocks nor log segments found. Creating new log.
I20260926 07:42:59.927245 26844 log.cc:824] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Log is configured to *not* fsync() on all Append() calls
I20260926 07:42:59.930253 26844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: No bootstrap required, opened a new log
Sep 26 07:42:59 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:42:59.939352 26844 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:42:59.939745 26844 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:42:59.939882 26844 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73e5bea656743159c18152df1e340ee, State: Initialized, Role: FOLLOWER
I20260926 07:42:59.940568 26844 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:42:59.940857 26844 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:42:59.941077 26844 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:42:59.941311 26844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:42:59.943534 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:42:59.921022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.11.9.126:46499)
0926 07:42:59.921865 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0926 07:42:59.921876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:42:59.921917 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:42:59.923693 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:42:59.923716 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:42:59.924111 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:42:59.924428 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:42:59.926198 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:59.927636 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:59.929571 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:42:59.930216 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:42:59.930399 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:42:59.935021 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:42:59.935028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:42:59.935059 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:42:59.935107 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:42:59.938661 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:59.939417 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:59.939424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:59.939431 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:59.939610 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:42:59.940076 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:42:59.940080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:42:59.940083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:42:59.940804 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:42:59.941149 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 07:42:59.941696 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 07:42:59.942149 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":418,"threads_started":1}
I20260926 07:42:59.944702 26844 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:42:59.945300 26844 leader_election.cc:302] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73e5bea656743159c18152df1e340ee; no voters:
I20260926 07:42:59.945962 26844 leader_election.cc:288] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:42:59.946192 26849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:42:59.947794 26849 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 LEADER]: Becoming Leader. State: Replica: f73e5bea656743159c18152df1e340ee, State: Running, Role: LEADER
I20260926 07:42:59.947981 26844 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:42:59.948428 26849 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:42:59.951730 26850 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:42:59.952241 26850 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:59.951725 26851 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73e5bea656743159c18152df1e340ee. Latest consensus state: current_term: 1 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:42:59.953094 26851 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:42:59.954159 26858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:42:59.964987 26858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:42:59.974584 26858 catalog_manager.cc:1347] Generated new cluster ID: 85519c9e83d34620808867268525f5b7
I20260926 07:42:59.974720 26858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:42:59.998543 26858 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:43:00.000957 26858 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:43:00.016232 26858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Generated new TSK 0
I20260926 07:43:00.017217 26858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:43:00.084266 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:43:00.425585 26870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:00.425871 26870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:00.426002 26870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:00.433125 26870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:00.433301 26870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:00.433490 26870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:43:00.443179 26870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:00.444461 26870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:00.445849 26870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:00.459537 26878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:00.459579 26876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:00.460150 26875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:00.462594 26870 server_base.cc:1034] running on GCE node
I20260926 07:43:00.463456 26870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:00.464649 26870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:00.465883 26870 hybrid_clock.cc:648] HybridClock initialized: now 1790408580465797 us; error 72 us; skew 500 ppm
I20260926 07:43:00.466270 26870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:00 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408580, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:00.471066 26870 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:43:00.473629 26870 webserver.cc:460] Webserver started at http://127.11.9.65:38275/ using document root <none> and password file <none>
I20260926 07:43:00.474651 26870 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:00.474798 26870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:00.475191 26870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:43:00.478657 26870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "c75dd0db81a440dd88fefca00a1dde68"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "aacf83c77306f208534e30ab367b4a7d"
server_key_iv: "138e600d60cd2309611c299061f61824"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.479548 26870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "c75dd0db81a440dd88fefca00a1dde68"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "aacf83c77306f208534e30ab367b4a7d"
server_key_iv: "138e600d60cd2309611c299061f61824"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.484794 26870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:43:00.488387 26885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:00.489738 26870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:43:00.489913 26870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "c75dd0db81a440dd88fefca00a1dde68"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "aacf83c77306f208534e30ab367b4a7d"
server_key_iv: "138e600d60cd2309611c299061f61824"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.490123 26870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:00.503522 26870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:00.507349 26870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:00.507632 26870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:00.508499 26870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:00.509815 26870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:00.509917 26870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:00.510063 26870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:00.510141 26870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:00.542678 26870 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:45039
I20260926 07:43:00.542760 26997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:45039 every 8 connection(s)
I20260926 07:43:00.543911 26870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:43:00.547844 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 26870
I20260926 07:43:00.548062 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:43:00.548873 11301 external_mini_cluster.cc:1428] Setting key 80e5a9ed592cd82279641a811c516057
Sep 26 07:43:00 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408580, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:00.567684 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:00.546436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:49405 (local address 127.11.9.126:46499)
0926 07:43:00.546901 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:00.546907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:00.548066 (+ 1159us) server_negotiation.cc:407] Connection header received
0926 07:43:00.550360 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:00.550366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:00.550440 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:00.550558 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:00.552573 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:00.553540 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:00.555958 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:00.556334 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:00.556430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:00.560758 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:00.560767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:00.560775 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:00.560819 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:00.564385 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:00.565058 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:00.565066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:00.565072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:00.565251 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:00.565663 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:00.565671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:00.565677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:00.566074 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:00.566224 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:00.566880 (+ 656us) server_negotiation.cc:299] Negotiation successful
0926 07:43:00.567167 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":216,"threads_started":1}
I20260926 07:43:00.568228 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:00.547000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:49405)
0926 07:43:00.547862 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:00.547919 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:00.550116 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:00.550831 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:00.550839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:00.551569 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:00.552388 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:00.552416 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:00.553693 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:00.553702 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:00.555820 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:00.555838 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:00.556554 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:00.556563 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:00.556870 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:00.557632 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:00.557666 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:00.560544 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:00.564532 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:00.564539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:00.564551 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:00.564940 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:00.565359 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:00.565363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:00.565365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:00.565551 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:00.566206 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:00.566211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:00.566509 (+ 298us) client_negotiation.cc:769] Sending connection context
0926 07:43:00.566724 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 07:43:00.567128 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":303,"threads_started":1}
I20260926 07:43:00.569826 26998 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:00.570175 26998 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:00.571069 26998 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:00.573706 26809 ts_manager.cc:194] Registered new tserver with Master: c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039)
I20260926 07:43:00.575909 26809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:49405
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:43:00.604903 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:43:00.921612 27004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:00.921862 27004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:00.922031 27004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:00.929149 27004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:00.929365 27004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:00.929572 27004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:43:00.939168 27004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:00.940410 27004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:00.941758 27004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:00.955035 27009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:00.955044 27010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:00.954962 27012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:00.955950 27004 server_base.cc:1034] running on GCE node
I20260926 07:43:00.956514 27004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:00.957417 27004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:00.958616 27004 hybrid_clock.cc:648] HybridClock initialized: now 1790408580958569 us; error 38 us; skew 500 ppm
I20260926 07:43:00.958932 27004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:00 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408580, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:00.962229 27004 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:43:00.963984 27004 webserver.cc:460] Webserver started at http://127.11.9.66:44809/ using document root <none> and password file <none>
I20260926 07:43:00.964676 27004 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:00.964818 27004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:00.965153 27004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:43:00.968206 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "22e53040d37c467da3c924d1466df77c"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "8efa376a0117b82678b1b498159c458a"
server_key_iv: "ecf7de0719a9cb5de84def486bca0ec5"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.968971 27004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "22e53040d37c467da3c924d1466df77c"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "8efa376a0117b82678b1b498159c458a"
server_key_iv: "ecf7de0719a9cb5de84def486bca0ec5"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.974706 27004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:43:00.978070 27019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:00.979319 27004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:43:00.979486 27004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "22e53040d37c467da3c924d1466df77c"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "8efa376a0117b82678b1b498159c458a"
server_key_iv: "ecf7de0719a9cb5de84def486bca0ec5"
server_key_version: "encryptionkey@0"
I20260926 07:43:00.979681 27004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:00.996126 27004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:01.000098 27004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:01.000385 27004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:01.001400 27004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:01.002684 27004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:01.002831 27004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:01.003023 27004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:01.003104 27004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:01.037859 27004 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:46587
I20260926 07:43:01.037899 27131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:46587 every 8 connection(s)
I20260926 07:43:01.039194 27004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:43:01.045718 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27004
I20260926 07:43:01.045902 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:43:01.046334 11301 external_mini_cluster.cc:1428] Setting key a4d01d402b3d920c529b9eb23fb66fa0
Sep 26 07:43:01 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408580, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:01.060067 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.041856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:56769 (local address 127.11.9.126:46499)
0926 07:43:01.042064 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.042069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.043192 (+ 1123us) server_negotiation.cc:407] Connection header received
0926 07:43:01.044943 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.044949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.045044 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.045157 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:01.046736 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.047616 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.049234 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.049757 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.049893 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.053355 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:01.053360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:01.053364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:01.053394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:01.056806 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.057586 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.057594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.057600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.057757 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.058162 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.058169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.058175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.058534 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:01.058688 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.059348 (+ 660us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.059609 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 07:43:01.060622 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.042342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:56769)
0926 07:43:01.043037 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:01.043073 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:01.044687 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:01.045403 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:01.045412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:01.045938 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:01.046573 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.046592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.047768 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.047773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.049068 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.049079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.049992 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.049999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.050226 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.050887 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:01.050908 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:01.053097 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:01.056948 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:01.056954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:01.056967 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:01.057460 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:01.057861 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:01.057864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:01.057866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:01.058045 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:01.058648 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:01.058654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:01.058968 (+ 314us) client_negotiation.cc:769] Sending connection context
0926 07:43:01.059197 (+ 229us) client_negotiation.cc:240] Negotiation successful
0926 07:43:01.059554 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":256,"threads_started":1}
I20260926 07:43:01.062098 27132 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:01.062418 27132 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:01.063288 27132 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:01.064821 26809 ts_manager.cc:194] Registered new tserver with Master: 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587)
I20260926 07:43:01.065685 26809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:56769
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:43:01.100320 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:43:01.418150 27137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:01.418403 27137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:01.418494 27137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:01.425400 27137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:01.425498 27137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:01.425606 27137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:43:01.434787 27137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:01.436484 27137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:01.437821 27137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:01.450861 27143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:01.450958 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:01.450964 27142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:01.451737 27137 server_base.cc:1034] running on GCE node
I20260926 07:43:01.452248 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:01.453295 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:01.454505 27137 hybrid_clock.cc:648] HybridClock initialized: now 1790408581454466 us; error 35 us; skew 500 ppm
I20260926 07:43:01.454757 27137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:01 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:01.457890 27137 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:43:01.459273 27137 webserver.cc:460] Webserver started at http://127.11.9.67:42037/ using document root <none> and password file <none>
I20260926 07:43:01.459928 27137 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:01.460068 27137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:01.460366 27137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:43:01.463150 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "87a380b8d02243889c2bc526a00bff13"
format_stamp: "Formatted at 2026-09-26 07:43:01 on dist-test-slave-x492"
server_key: "cfbd767d3fe38ec22b289724917f2332"
server_key_iv: "59cc297c4f9f0ce6541d25cafb2380e2"
server_key_version: "encryptionkey@0"
I20260926 07:43:01.463881 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "87a380b8d02243889c2bc526a00bff13"
format_stamp: "Formatted at 2026-09-26 07:43:01 on dist-test-slave-x492"
server_key: "cfbd767d3fe38ec22b289724917f2332"
server_key_iv: "59cc297c4f9f0ce6541d25cafb2380e2"
server_key_version: "encryptionkey@0"
I20260926 07:43:01.468645 27137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:43:01.472298 27152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:01.473704 27137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:43:01.473868 27137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "87a380b8d02243889c2bc526a00bff13"
format_stamp: "Formatted at 2026-09-26 07:43:01 on dist-test-slave-x492"
server_key: "cfbd767d3fe38ec22b289724917f2332"
server_key_iv: "59cc297c4f9f0ce6541d25cafb2380e2"
server_key_version: "encryptionkey@0"
I20260926 07:43:01.474073 27137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:01.494650 27137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:01.498565 27137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:01.498847 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:01.499774 27137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:01.501228 27137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:01.501338 27137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 07:43:01.501462 27137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:01.501547 27137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:01.534245 27137 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38355
I20260926 07:43:01.534330 27264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38355 every 8 connection(s)
I20260926 07:43:01.535501 27137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:43:01.542004 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27137
I20260926 07:43:01.542187 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:43:01.542722 11301 external_mini_cluster.cc:1428] Setting key e5975c5715c9a4e80102bd0ebb550918
Sep 26 07:43:01 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:01.555404 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.537986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:54667 (local address 127.11.9.126:46499)
0926 07:43:01.538196 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.538203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.539334 (+ 1131us) server_negotiation.cc:407] Connection header received
0926 07:43:01.541072 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.541076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.541155 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.541300 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:01.542799 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.543691 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.545896 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.546290 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.546385 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.550229 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:01.550234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:01.550237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:01.550270 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:01.552793 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.553522 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.553527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.553530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.553665 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.554028 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.554031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.554033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.554292 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:01.554402 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.554995 (+ 593us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.555152 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:43:01.556399 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.538482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:54667)
0926 07:43:01.539171 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:01.539212 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:01.540781 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:01.541476 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:01.541485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:01.541949 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:01.542627 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.542648 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.543829 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.543835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.545727 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.545741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.546492 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.546499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.546763 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.547461 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:01.547488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:01.550048 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:01.552906 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:01.552912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:01.552922 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:01.553415 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:01.553767 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:01.553770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:01.553772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:01.553929 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:01.554392 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:01.554397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:01.554686 (+ 289us) client_negotiation.cc:769] Sending connection context
0926 07:43:01.554893 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 07:43:01.555223 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":275,"threads_started":1}
I20260926 07:43:01.557826 27265 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:01.558132 27265 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:01.558815 27265 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:01.560142 26809 ts_manager.cc:194] Registered new tserver with Master: 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:01.560896 26809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:54667
I20260926 07:43:01.570333 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:43:01.579172 26998 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:01.587069 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.574724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.11.9.126:46499)
0926 07:43:01.574881 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.574885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.575274 (+ 389us) server_negotiation.cc:407] Connection header received
0926 07:43:01.575409 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.575412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.575473 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.575554 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:01.576999 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.577877 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.579145 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.579726 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.579871 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.582338 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:01.582343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:01.582346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:01.582374 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:01.584886 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.585559 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.585563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.585565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.585671 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:01.585987 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:01.585990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:01.585992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:01.586284 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:01.586461 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.586684 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.586826 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:43:01.590739 26809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:43:01.597592 26809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:43:01.632525 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.609011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.0.0.1:59960)
0926 07:43:01.623097 (+ 14086us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:01.623113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:01.623236 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:01.624324 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:01.624331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:01.624360 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:01.624771 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.624786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.627223 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.627231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.629451 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.629464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.631526 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.631530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.631705 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.631741 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:43:01.631884 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 07:43:01.632094 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13964,"mutex_wait_us":50,"thread_start_us":12505,"threads_started":1}
I20260926 07:43:01.632483 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.609552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.11.9.67:38355)
0926 07:43:01.623822 (+ 14270us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.623832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.623866 (+ 34us) server_negotiation.cc:407] Connection header received
0926 07:43:01.623943 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.623948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.624131 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.624318 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:01.625023 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.627081 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.629671 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.631126 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.631243 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.631415 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.631943 (+ 528us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.632099 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14101,"thread_start_us":13946,"threads_started":1}
I20260926 07:43:01.634653 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.608860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:41894)
0926 07:43:01.621968 (+ 13108us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:01.622015 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:01.622214 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:01.625512 (+ 3298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:01.625517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:01.625542 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:01.626391 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.626402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.628736 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.628742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.631033 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.631051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.633867 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.633873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.634035 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.634064 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 07:43:01.634154 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 07:43:01.634290 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12785,"mutex_wait_us":68,"thread_start_us":13969,"threads_started":1}
I20260926 07:43:01.635350 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.609109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.11.9.65:45039)
0926 07:43:01.625018 (+ 15909us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.625024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.625043 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:43:01.625107 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.625112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.625319 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.625530 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:01.627244 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.628604 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.631207 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.633399 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.633531 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.633726 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.634872 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.635041 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15760,"thread_start_us":15597,"threads_started":1}
I20260926 07:43:01.636420 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.609481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46587 (local address 127.0.0.1:33674)
0926 07:43:01.626908 (+ 17427us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:01.626922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:01.627047 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:01.627669 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:01.627676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:01.627714 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:01.628148 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.628159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.629720 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.629725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.631814 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:01.631828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.635689 (+ 3861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.635695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:01.635854 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.635883 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 07:43:01.635977 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 07:43:01.636102 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17305,"mutex_wait_us":5,"thread_start_us":17185,"threads_started":1}
I20260926 07:43:01.636457 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:01.609763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.11.9.66:46587)
0926 07:43:01.622639 (+ 12876us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:01.622652 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:01.627028 (+ 4376us) server_negotiation.cc:407] Connection header received
0926 07:43:01.627240 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:01.627248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:01.627476 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:01.627669 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:01.628321 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.629536 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.631983 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:01.633432 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:01.633520 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:01.633698 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:01.636052 (+ 2354us) server_negotiation.cc:299] Negotiation successful
0926 07:43:01.636223 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12700,"thread_start_us":12587,"threads_started":1}
I20260926 07:43:01.637081 27200 tablet_service.cc:1467] Processing CreateTablet for tablet 8493a32f036c4d52abb032ea4109a2fd (DEFAULT_TABLE table=test-table [id=555fd8d897e54dbfad052edf77fccdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:01.638587 27200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8493a32f036c4d52abb032ea4109a2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:01.639205 26933 tablet_service.cc:1467] Processing CreateTablet for tablet 8493a32f036c4d52abb032ea4109a2fd (DEFAULT_TABLE table=test-table [id=555fd8d897e54dbfad052edf77fccdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:01.640652 26933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8493a32f036c4d52abb032ea4109a2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:01.640268 27067 tablet_service.cc:1467] Processing CreateTablet for tablet 8493a32f036c4d52abb032ea4109a2fd (DEFAULT_TABLE table=test-table [id=555fd8d897e54dbfad052edf77fccdfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:01.641578 27067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8493a32f036c4d52abb032ea4109a2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:01.652335 27283 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Bootstrap starting.
I20260926 07:43:01.655378 27283 tablet_bootstrap.cc:654] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:01.655982 27284 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Bootstrap starting.
I20260926 07:43:01.656852 27283 log.cc:824] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:01.657819 27285 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Bootstrap starting.
I20260926 07:43:01.660194 27283 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: No bootstrap required, opened a new log
I20260926 07:43:01.661506 27285 tablet_bootstrap.cc:654] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:01.662528 27284 tablet_bootstrap.cc:654] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:01.662699 27285 log.cc:824] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:01.663617 27283 ts_tablet_manager.cc:1397] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260926 07:43:01.663713 27284 log.cc:824] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:01.665489 27285 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: No bootstrap required, opened a new log
I20260926 07:43:01.665735 27285 ts_tablet_manager.cc:1397] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 07:43:01.666513 27284 tablet_bootstrap.cc:492] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: No bootstrap required, opened a new log
I20260926 07:43:01.666752 27284 ts_tablet_manager.cc:1397] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260926 07:43:01.673439 27285 raft_consensus.cc:348] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.673799 27285 raft_consensus.cc:374] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:01.673899 27285 raft_consensus.cc:729] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e53040d37c467da3c924d1466df77c, State: Initialized, Role: FOLLOWER
I20260926 07:43:01.673892 27283 raft_consensus.cc:348] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.674304 27283 raft_consensus.cc:374] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:01.674394 27283 raft_consensus.cc:729] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a380b8d02243889c2bc526a00bff13, State: Initialized, Role: FOLLOWER
I20260926 07:43:01.674539 27285 consensus_queue.cc:260] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.674959 27283 consensus_queue.cc:260] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.675798 27132 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:01.676211 27285 ts_tablet_manager.cc:1428] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260926 07:43:01.678131 27265 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:01.678710 27283 ts_tablet_manager.cc:1428] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 07:43:01.679111 27284 raft_consensus.cc:348] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.679574 27284 raft_consensus.cc:374] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:01.679683 27284 raft_consensus.cc:729] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75dd0db81a440dd88fefca00a1dde68, State: Initialized, Role: FOLLOWER
I20260926 07:43:01.680266 27284 consensus_queue.cc:260] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:01.682921 27284 ts_tablet_manager.cc:1428] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260926 07:43:01.787147 27266 tablet.cc:2369] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:43:01.791364 27133 tablet.cc:2369] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:43:01.796099 26999 tablet.cc:2369] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:43:02.092042 27289 raft_consensus.cc:482] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:02.092309 27289 raft_consensus.cc:504] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:02.094079 27289 leader_election.cc:288] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:02.096251 27290 raft_consensus.cc:482] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:02.096597 27290 raft_consensus.cc:504] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:02.104003 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.094808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:42289 (local address 127.11.9.67:38355)
0926 07:43:02.095042 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.095048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.095499 (+ 451us) server_negotiation.cc:407] Connection header received
0926 07:43:02.095672 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.095678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.095757 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.095853 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.096432 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.097593 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.100582 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.102453 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.102580 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.102721 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.103437 (+ 716us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.103612 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 07:43:02.105005 27290 leader_election.cc:288] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039), 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587)
I20260926 07:43:02.105257 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.094370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.11.9.66:42289)
0926 07:43:02.095373 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.095392 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.095537 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.095881 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.095885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.095908 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.096264 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.096280 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.097736 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.097741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.100364 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.100384 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.102940 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.102949 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.103207 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.103256 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.104376 (+ 1120us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.104577 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":71,"thread_start_us":878,"threads_started":1}
I20260926 07:43:02.111119 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.094522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:41749 (local address 127.11.9.65:45039)
0926 07:43:02.094792 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.094802 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.095670 (+ 868us) server_negotiation.cc:407] Connection header received
0926 07:43:02.095852 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.095857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.095927 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.096045 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.096755 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.097626 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.107880 (+ 10254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.109689 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.109858 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.110013 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.110663 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.110821 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:43:02.111955 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.094514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.11.9.66:41749)
0926 07:43:02.095541 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.095560 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.095706 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.096192 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.096199 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.096230 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.096609 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.096618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.097777 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.097782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.107669 (+ 9887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.107691 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.110189 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.110198 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.110447 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.110496 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.111436 (+ 940us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.111571 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":45,"thread_start_us":466,"threads_started":1}
I20260926 07:43:02.112063 27220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a380b8d02243889c2bc526a00bff13" is_pre_election: true
I20260926 07:43:02.112567 27220 raft_consensus.cc:2454] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 0.
I20260926 07:43:02.113097 27291 raft_consensus.cc:482] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:02.113452 26953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75dd0db81a440dd88fefca00a1dde68" is_pre_election: true
I20260926 07:43:02.113837 27020 leader_election.cc:302] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13; no voters:
I20260926 07:43:02.113473 27291 raft_consensus.cc:504] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:02.115656 27291 leader_election.cc:288] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355), 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587)
I20260926 07:43:02.115885 27289 raft_consensus.cc:2781] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:43:02.116044 27289 raft_consensus.cc:482] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:43:02.116181 27289 raft_consensus.cc:3037] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:02.118743 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.099302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:33553 (local address 127.11.9.66:46587)
0926 07:43:02.099535 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.099543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.111021 (+ 11478us) server_negotiation.cc:407] Connection header received
0926 07:43:02.111141 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.111146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.111217 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.111306 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.112549 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.113569 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.115997 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.117346 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.117466 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.117562 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.118348 (+ 786us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.118489 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:43:02.119540 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.098904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46587 (local address 127.11.9.67:33553)
0926 07:43:02.110906 (+ 12002us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.110924 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.111038 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.111706 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.111711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.111754 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.112365 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.112382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.113708 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.113720 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.115805 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.115820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.117967 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.117972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.118157 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.118188 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.118951 (+ 763us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.119084 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11901,"mutex_wait_us":25,"spinlock_wait_cycles":26880,"thread_start_us":11538,"threads_started":1}
I20260926 07:43:02.120141 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.098668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.11.9.67:50657)
0926 07:43:02.110172 (+ 11504us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.110195 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.110344 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.111754 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.111758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.111776 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.112271 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.112777 (+ 506us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.115221 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.115227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.117330 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.117343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.119303 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.119309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.119471 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.119514 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.119611 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.119745 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11331,"mutex_wait_us":2,"spinlock_wait_cycles":16640,"thread_start_us":12455,"threads_started":1}
I20260926 07:43:02.120410 27087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "87a380b8d02243889c2bc526a00bff13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22e53040d37c467da3c924d1466df77c" is_pre_election: true
I20260926 07:43:02.120456 27289 raft_consensus.cc:504] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:02.120918 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.099032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:50657 (local address 127.11.9.65:45039)
0926 07:43:02.111204 (+ 12172us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.111209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.111229 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:43:02.111285 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.111292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.111360 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.111465 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.113001 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.115054 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.117519 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.118912 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.119030 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.119146 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.120465 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.120605 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12045,"thread_start_us":11749,"threads_started":1}
I20260926 07:43:02.121018 27087 raft_consensus.cc:2379] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87a380b8d02243889c2bc526a00bff13 in current term 1: Already voted for candidate 22e53040d37c467da3c924d1466df77c in this term.
I20260926 07:43:02.121743 26952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "87a380b8d02243889c2bc526a00bff13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75dd0db81a440dd88fefca00a1dde68" is_pre_election: true
I20260926 07:43:02.122143 26953 raft_consensus.cc:2454] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 0.
I20260926 07:43:02.124167 26953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75dd0db81a440dd88fefca00a1dde68"
I20260926 07:43:02.124411 26953 raft_consensus.cc:3037] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:02.124687 27154 leader_election.cc:302] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87a380b8d02243889c2bc526a00bff13; no voters: 22e53040d37c467da3c924d1466df77c, c75dd0db81a440dd88fefca00a1dde68
I20260926 07:43:02.125527 27290 raft_consensus.cc:3037] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:02.129033 26953 raft_consensus.cc:2454] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 1.
I20260926 07:43:02.129669 27023 leader_election.cc:302] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e53040d37c467da3c924d1466df77c, c75dd0db81a440dd88fefca00a1dde68; no voters:
I20260926 07:43:02.129719 27290 raft_consensus.cc:2726] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:43:02.130719 27289 leader_election.cc:288] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 election: Requested vote from peers c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:02.130980 27289 raft_consensus.cc:2781] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:43:02.131145 27289 raft_consensus.cc:686] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 LEADER]: Becoming Leader. State: Replica: 22e53040d37c467da3c924d1466df77c, State: Running, Role: LEADER
I20260926 07:43:02.131793 27289 consensus_queue.cc:237] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } }
I20260926 07:43:02.133131 27220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a380b8d02243889c2bc526a00bff13"
I20260926 07:43:02.137514 27220 raft_consensus.cc:2454] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 1.
I20260926 07:43:02.142498 26809 catalog_manager.cc:5495] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c reported cstate change: term changed from 0 to 1, leader changed from <none> to 22e53040d37c467da3c924d1466df77c (127.11.9.66). New cstate: current_term: 1 leader_uuid: "22e53040d37c467da3c924d1466df77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } health_report { overall_health: HEALTHY } } }
I20260926 07:43:02.146498 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.121712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46587 (local address 127.11.9.65:35207)
0926 07:43:02.139179 (+ 17467us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.139200 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.139460 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.139995 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.140000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.140031 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.140481 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.140496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.141971 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.141978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.144260 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.144275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.145587 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.145596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.145787 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.145826 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.145938 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.146078 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17347,"mutex_wait_us":45,"thread_start_us":16781,"threads_started":1}
I20260926 07:43:02.146522 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.123885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:35207 (local address 127.11.9.66:46587)
0926 07:43:02.129343 (+ 5458us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.129351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.139286 (+ 9935us) server_negotiation.cc:407] Connection header received
0926 07:43:02.139631 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.139636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.139733 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.139796 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.140686 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.141839 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.144468 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.145365 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.145505 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.145609 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.145978 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.146149 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5295}
I20260926 07:43:02.146704 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.117682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:51553 (local address 127.11.9.67:38355)
0926 07:43:02.117815 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.117820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.139026 (+ 21206us) server_negotiation.cc:407] Connection header received
0926 07:43:02.139309 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.139344 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.139453 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.139552 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:02.140356 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.141458 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.144422 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.145544 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.145672 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.145786 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.146292 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.146424 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 07:43:02.147516 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.115757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.11.9.65:51553)
0926 07:43:02.138911 (+ 23154us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:02.138937 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:02.139152 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:02.139713 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:02.139718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:02.139741 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:02.140129 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.140146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.141641 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.141646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.144260 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:02.144275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.145952 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.145957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:02.146108 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.146135 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 07:43:02.146866 (+ 731us) client_negotiation.cc:240] Negotiation successful
0926 07:43:02.147144 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22954,"mutex_wait_us":157,"thread_start_us":23163,"threads_started":1}
I20260926 07:43:02.147717 27087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "c75dd0db81a440dd88fefca00a1dde68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22e53040d37c467da3c924d1466df77c" is_pre_election: true
I20260926 07:43:02.148157 27220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8493a32f036c4d52abb032ea4109a2fd" candidate_uuid: "c75dd0db81a440dd88fefca00a1dde68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a380b8d02243889c2bc526a00bff13" is_pre_election: true
I20260926 07:43:02.148454 27220 raft_consensus.cc:2379] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c75dd0db81a440dd88fefca00a1dde68 in current term 1: Already voted for candidate 22e53040d37c467da3c924d1466df77c in this term.
I20260926 07:43:02.149061 26886 leader_election.cc:302] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:43:02.149525 27291 raft_consensus.cc:2726] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:43:02.197934 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.191791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.11.9.66:46587)
0926 07:43:02.192025 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.192034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.192469 (+ 435us) server_negotiation.cc:407] Connection header received
0926 07:43:02.192733 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.192738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.192816 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.192974 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:43:02.193524 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.194230 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.195754 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.196184 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.196326 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.196803 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:43:02.197337 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:43:02.197511 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.197565 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.197645 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:43:02.205860 27220 raft_consensus.cc:1261] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Refusing update from remote peer 22e53040d37c467da3c924d1466df77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:43:02.205843 26953 raft_consensus.cc:1261] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Refusing update from remote peer 22e53040d37c467da3c924d1466df77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:43:02.206867 27299 consensus_queue.cc:1035] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:43:02.207233 27289 consensus_queue.cc:1035] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:43:02.220263 27308 mvcc.cc:205] Tried to move back new op lower bound from 7333513552705462272 to 7333513552437555200. Current Snapshot: MvccSnapshot[applied={T|T < 7333513552705462272}]
I20260926 07:43:02.230417 26808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:43:02.230679 26808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:43:02.234302 26808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Sending DeleteTablet for 3 replicas of tablet 8493a32f036c4d52abb032ea4109a2fd
I20260926 07:43:02.235467 26933 tablet_service.cc:1514] Processing DeleteTablet for tablet 8493a32f036c4d52abb032ea4109a2fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:02 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:41894
I20260926 07:43:02.235532 27200 tablet_service.cc:1514] Processing DeleteTablet for tablet 8493a32f036c4d52abb032ea4109a2fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:02 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:59960
I20260926 07:43:02.235692 27067 tablet_service.cc:1514] Processing DeleteTablet for tablet 8493a32f036c4d52abb032ea4109a2fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:02 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:33674
I20260926 07:43:02.236126 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26870
I20260926 07:43:02.253906 27318 tablet_replica.cc:330] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: stopping tablet replica
I20260926 07:43:02.254487 27318 raft_consensus.cc:2229] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:43:02.255316 27318 raft_consensus.cc:2258] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:02.257606 27317 tablet_replica.cc:330] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: stopping tablet replica
I20260926 07:43:02.258034 27317 raft_consensus.cc:2229] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:43:02.258520 27317 raft_consensus.cc:2258] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:02.258702 27318 ts_tablet_manager.cc:1905] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:43:02.260273 27317 ts_tablet_manager.cc:1905] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:43:02.265694 27316 tablet_replica.cc:330] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: stopping tablet replica
I20260926 07:43:02.266062 27316 raft_consensus.cc:2229] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:43:02.266474 27316 raft_consensus.cc:2258] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:02.267891 27317 ts_tablet_manager.cc:1918] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:43:02.268148 27317 log.cc:1196] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/8493a32f036c4d52abb032ea4109a2fd
I20260926 07:43:02.268682 27317 ts_tablet_manager.cc:1939] T 8493a32f036c4d52abb032ea4109a2fd P 87a380b8d02243889c2bc526a00bff13: Deleting consensus metadata
I20260926 07:43:02.270538 26793 catalog_manager.cc:4841] TS 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): tablet 8493a32f036c4d52abb032ea4109a2fd (table test-table [id=555fd8d897e54dbfad052edf77fccdfb]) successfully deleted
I20260926 07:43:02.271281 27316 ts_tablet_manager.cc:1905] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 07:43:02 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:43:02.274351 27318 ts_tablet_manager.cc:1918] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:43:02.274523 27318 log.cc:1196] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/8493a32f036c4d52abb032ea4109a2fd
I20260926 07:43:02.275077 27318 ts_tablet_manager.cc:1939] T 8493a32f036c4d52abb032ea4109a2fd P 22e53040d37c467da3c924d1466df77c: Deleting consensus metadata
I20260926 07:43:02.276865 26794 catalog_manager.cc:4841] TS 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): tablet 8493a32f036c4d52abb032ea4109a2fd (table test-table [id=555fd8d897e54dbfad052edf77fccdfb]) successfully deleted
I20260926 07:43:02.280098 27316 ts_tablet_manager.cc:1918] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:43:02.280272 27316 log.cc:1196] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/8493a32f036c4d52abb032ea4109a2fd
I20260926 07:43:02.280786 27316 ts_tablet_manager.cc:1939] T 8493a32f036c4d52abb032ea4109a2fd P c75dd0db81a440dd88fefca00a1dde68: Deleting consensus metadata
I20260926 07:43:02.281418 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.237413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.11.9.65:45039)
0926 07:43:02.237665 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.237675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.253902 (+ 16227us) server_negotiation.cc:407] Connection header received
0926 07:43:02.254239 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.254249 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.254370 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.254463 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:02.257256 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.258742 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.261570 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.262495 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.262618 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.273036 (+ 10418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:02.273050 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:02.273067 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:02.273123 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:02.277755 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.278782 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.278792 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.278796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.278994 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.279457 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.279462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.279466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.279948 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:02.280420 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.280670 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.280974 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 07:43:02.282042 26993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:02.282586 26796 catalog_manager.cc:4841] TS c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039): tablet 8493a32f036c4d52abb032ea4109a2fd (table test-table [id=555fd8d897e54dbfad052edf77fccdfb]) successfully deleted
I20260926 07:43:02.374771 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26870
I20260926 07:43:02.393630 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27004
Sep 26 07:43:02 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:43:02.412237 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.394697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.11.9.66:46587)
0926 07:43:02.394927 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.394935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.394967 (+ 32us) server_negotiation.cc:407] Connection header received
0926 07:43:02.395039 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.395046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.395130 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.395257 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:02.396498 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.397865 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.399297 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.400118 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.400275 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.404936 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:02.404954 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:02.404976 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:02.405026 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:02.409240 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.409934 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.409945 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.409950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.410138 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.410477 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.410483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.410487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.410989 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:02.411381 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.411545 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.411797 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 07:43:02.413056 27127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:02.526320 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27004
I20260926 07:43:02.544616 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27137
Sep 26 07:43:02 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:43:02.560911 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:02.545366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.11.9.67:38355)
0926 07:43:02.545572 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:02.545576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:02.545664 (+ 88us) server_negotiation.cc:407] Connection header received
0926 07:43:02.545822 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:02.545826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:02.545882 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:02.545959 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:02.547628 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.548440 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.550359 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:02.550797 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:02.550890 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:02.555840 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:02.555851 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:02.555864 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:02.555894 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:02.558659 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.559213 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.559218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.559221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.559338 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:02.559627 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:02.559630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:02.559631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:02.559924 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:02.560142 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:02.560419 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 07:43:02.560593 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:43:02.561621 27260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:02.656224 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27137
I20260926 07:43:02.673102 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 26777
I20260926 07:43:02.674263 26839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:02.783648 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 26777
I20260926 07:43:22.805331 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--webserver_interface=127.11.9.126
--webserver_port=33745
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:46499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:43:23.127305 27324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:23.127630 27324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:23.127801 27324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:23.134754 27324 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 07:43:23.134886 27324 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 07:43:23.134981 27324 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 07:43:23.135077 27324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:43:23.135156 27324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:23.135275 27324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:23.135363 27324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:43:23.135443 27324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:43:23.144788 27324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:46499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=33745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:23.146039 27324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:23.147493 27324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:23.169921 27333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:23.169973 27331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:23.170032 27330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:23.170361 27324 server_base.cc:1034] running on GCE node
I20260926 07:43:23.171590 27324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:23.173270 27324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:23.174525 27324 hybrid_clock.cc:648] HybridClock initialized: now 1790408603174456 us; error 58 us; skew 500 ppm
I20260926 07:43:23.174904 27324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:23 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:23.188488 27324 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:43:23.190218 27324 webserver.cc:460] Webserver started at http://127.11.9.126:33745/ using document root <none> and password file <none>
I20260926 07:43:23.190974 27324 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:23.191078 27324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:23.197324 27324 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 07:43:23.200274 27340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:23.201736 27324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:43:23.201920 27324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "f73e5bea656743159c18152df1e340ee"
format_stamp: "Formatted at 2026-09-26 07:42:59 on dist-test-slave-x492"
server_key: "a049d55d3a18ad022ecceb02cee12734"
server_key_iv: "b396b8bd3cefb5b81042d068b83b048e"
server_key_version: "encryptionkey@0"
I20260926 07:43:23.202962 27324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:23.221079 27324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:23.225034 27324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:23.225375 27324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:23.244057 27324 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:46499
I20260926 07:43:23.244089 27391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:46499 every 8 connection(s)
I20260926 07:43:23.245579 27324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:43:23.247546 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27324
I20260926 07:43:23.248327 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:45039
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=38275
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:43:23.252560 27392 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:43:23.255796 27392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap starting.
I20260926 07:43:23.275629 27392 log.cc:824] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:23.287305 27392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:43:23.287853 27392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap complete.
I20260926 07:43:23.296928 27392 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:23.297950 27392 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73e5bea656743159c18152df1e340ee, State: Initialized, Role: FOLLOWER
I20260926 07:43:23.298645 27392 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:23.298974 27392 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:43:23.299109 27392 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:43:23.299316 27392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:43:23.303114 27392 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:23.303627 27392 leader_election.cc:302] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73e5bea656743159c18152df1e340ee; no voters:
I20260926 07:43:23.304320 27392 leader_election.cc:288] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 07:43:23.304467 27396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:43:23.307220 27392 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:43:23.311898 27396 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 LEADER]: Becoming Leader. State: Replica: f73e5bea656743159c18152df1e340ee, State: Running, Role: LEADER
I20260926 07:43:23.312572 27396 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:23.315536 27397 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:43:23.315917 27397 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:43:23.316500 27398 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73e5bea656743159c18152df1e340ee. Latest consensus state: current_term: 2 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:43:23.316768 27398 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:43:23.328357 27406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:43:23.338358 27406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:43:23.339792 27406 catalog_manager.cc:1259] Loaded cluster ID: 85519c9e83d34620808867268525f5b7
I20260926 07:43:23.339948 27406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:43:23.344165 27406 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:43:23.345724 27406 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Loaded TSK: 0
I20260926 07:43:23.353215 27413 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 07:43:23.377213 27406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Generated new TSK 1
I20260926 07:43:23.377614 27406 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 07:43:23.605615 27394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:23.606014 27394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:23.606186 27394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:23.613802 27394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:23.613945 27394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:23.614115 27394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:43:23.623862 27394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:45039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=38275
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:23.625089 27394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:23.626410 27394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:23.641054 27420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:23.641191 27419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:23.641076 27422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:23.642656 27394 server_base.cc:1034] running on GCE node
I20260926 07:43:23.643278 27394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:23.644188 27394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:23.645447 27394 hybrid_clock.cc:648] HybridClock initialized: now 1790408603645397 us; error 52 us; skew 500 ppm
I20260926 07:43:23.645762 27394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:23 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:23.648917 27394 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:43:23.650373 27394 webserver.cc:460] Webserver started at http://127.11.9.65:38275/ using document root <none> and password file <none>
I20260926 07:43:23.651046 27394 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:23.651199 27394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:23.656500 27394 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:43:23.659390 27429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:23.660898 27394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:43:23.661078 27394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "c75dd0db81a440dd88fefca00a1dde68"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "aacf83c77306f208534e30ab367b4a7d"
server_key_iv: "138e600d60cd2309611c299061f61824"
server_key_version: "encryptionkey@0"
I20260926 07:43:23.662400 27394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:23.679206 27394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:23.683314 27394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:23.683599 27394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:23.684659 27394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:23.686121 27394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:23.686254 27394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:23.686374 27394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:23.686465 27394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:23.719321 27394 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:45039
I20260926 07:43:23.719408 27541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:45039 every 8 connection(s)
I20260926 07:43:23.721218 27394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:43:23.731467 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27394
I20260926 07:43:23.732221 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:46587
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=44809
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:43:23 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:23.746780 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:23.724639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:58487)
0926 07:43:23.725513 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:23.725557 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:23.727248 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:23.728435 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:23.728442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:23.729063 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:23.729654 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:23.729671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:23.731218 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:23.731224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:23.733340 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:23.733360 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:23.734374 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:23.734382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:23.734693 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:23.735411 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:23.735444 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:23.738450 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:23.742860 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:23.742866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:23.742877 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:23.743261 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:23.743770 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:23.743773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:23.743775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:23.743955 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:23.744859 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:23.744865 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:23.745131 (+ 266us) client_negotiation.cc:769] Sending connection context
0926 07:43:23.745436 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 07:43:23.745807 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":353,"threads_started":1}
I20260926 07:43:23.747365 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:23.724350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:58487 (local address 127.11.9.126:46499)
0926 07:43:23.724912 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:23.724924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:23.725673 (+ 749us) server_negotiation.cc:407] Connection header received
0926 07:43:23.727695 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:23.727707 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:23.728023 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:23.728303 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:23.729833 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:23.731044 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:23.733483 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:23.734130 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:23.734278 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:23.738734 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:23.738742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:23.738756 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:23.738797 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:23.742709 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:23.743415 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:23.743427 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:23.743437 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:23.743654 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:23.744077 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:23.744085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:23.744091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:23.744730 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:23.744925 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:23.745698 (+ 773us) server_negotiation.cc:299] Negotiation successful
0926 07:43:23.746152 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":228,"threads_started":1}
I20260926 07:43:23.750118 27542 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:23.750463 27542 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:23.751318 27542 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:23.753917 27357 ts_manager.cc:194] Registered new tserver with Master: c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039)
I20260926 07:43:23.755443 27357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:58487
W20260926 07:43:24.059449 27546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:24.059721 27546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:24.059888 27546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:24.067631 27546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:24.067770 27546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:24.067950 27546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:43:24.078178 27546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:46587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=44809
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:24.079476 27546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:24.080789 27546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:24.093628 27552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:24.093683 27554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:24.093659 27551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:24.094776 27546 server_base.cc:1034] running on GCE node
I20260926 07:43:24.095320 27546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:24.096149 27546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:24.097334 27546 hybrid_clock.cc:648] HybridClock initialized: now 1790408604097273 us; error 66 us; skew 500 ppm
I20260926 07:43:24.097599 27546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:24 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:24.100505 27546 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:43:24.101907 27546 webserver.cc:460] Webserver started at http://127.11.9.66:44809/ using document root <none> and password file <none>
I20260926 07:43:24.102560 27546 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:24.102720 27546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:24.108001 27546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:43:24.110642 27561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.111876 27546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:43:24.112048 27546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "22e53040d37c467da3c924d1466df77c"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "8efa376a0117b82678b1b498159c458a"
server_key_iv: "ecf7de0719a9cb5de84def486bca0ec5"
server_key_version: "encryptionkey@0"
I20260926 07:43:24.112985 27546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:24.131372 27546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:24.135082 27546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:24.135356 27546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:24.136323 27546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:24.137652 27546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:24.137774 27546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.137916 27546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:24.138005 27546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.171000 27546 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:46587
I20260926 07:43:24.171082 27673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:46587 every 8 connection(s)
I20260926 07:43:24.172485 27546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:43:24.179049 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27546
I20260926 07:43:24.179643 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:38355
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=42037
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 07:43:24 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:24.193326 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.175049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:36883 (local address 127.11.9.126:46499)
0926 07:43:24.175235 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.175240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.176366 (+ 1126us) server_negotiation.cc:407] Connection header received
0926 07:43:24.178237 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.178243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.178340 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.178454 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:24.180092 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.180789 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.182443 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.182863 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.183003 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.186653 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:24.186662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:24.186668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:24.186716 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:24.190115 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.190833 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.190839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.190844 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.190994 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.191395 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.191402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.191408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.191800 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:24.191948 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.192598 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.192858 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:43:24.193987 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.175505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:36883)
0926 07:43:24.176169 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:24.176206 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:24.177965 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:24.178689 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:24.178697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:24.179268 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:24.179920 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.179940 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.180958 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.180963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.182289 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.182301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.183107 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.183114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.183342 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.184025 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:24.184050 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:24.186392 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:24.190249 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:24.190256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:24.190269 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:24.190690 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:24.191101 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:24.191104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:24.191106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:24.191273 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:24.191925 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:24.191931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:24.192224 (+ 293us) client_negotiation.cc:769] Sending connection context
0926 07:43:24.192450 (+ 226us) client_negotiation.cc:240] Negotiation successful
0926 07:43:24.192823 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":264,"threads_started":1}
I20260926 07:43:24.195505 27674 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:24.195801 27674 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:24.196506 27674 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:24.197978 27357 ts_manager.cc:194] Registered new tserver with Master: 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587)
I20260926 07:43:24.198769 27357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:36883
W20260926 07:43:24.503846 27677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:24.504144 27677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:24.504315 27677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:24.511816 27677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:24.511973 27677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:24.512149 27677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:43:24.522063 27677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:38355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=42037
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:24.523396 27677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:24.524663 27677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:24.537671 27685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:24.537638 27683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:24.537709 27682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:24.542508 27677 server_base.cc:1034] running on GCE node
I20260926 07:43:24.543064 27677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:24.543967 27677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:24.545199 27677 hybrid_clock.cc:648] HybridClock initialized: now 1790408604545121 us; error 30 us; skew 500 ppm
I20260926 07:43:24.545516 27677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:24 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:24.548477 27677 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:43:24.549968 27677 webserver.cc:460] Webserver started at http://127.11.9.67:42037/ using document root <none> and password file <none>
I20260926 07:43:24.550664 27677 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:24.550820 27677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:24.555994 27677 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:43:24.558688 27692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.559957 27677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:43:24.560132 27677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "87a380b8d02243889c2bc526a00bff13"
format_stamp: "Formatted at 2026-09-26 07:43:01 on dist-test-slave-x492"
server_key: "cfbd767d3fe38ec22b289724917f2332"
server_key_iv: "59cc297c4f9f0ce6541d25cafb2380e2"
server_key_version: "encryptionkey@0"
I20260926 07:43:24.561087 27677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:24.590732 27677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:24.594799 27677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:24.595086 27677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:24.596050 27677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:24.597538 27677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:24.597675 27677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.597800 27677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:24.597908 27677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:24.630849 27677 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38355
I20260926 07:43:24.630939 27804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38355 every 8 connection(s)
I20260926 07:43:24.632289 27677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:43:24.635155 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27677
Sep 26 07:43:24 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:43:24.652619 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.636017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.11.9.126:46499)
0926 07:43:24.637537 (+ 1520us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.637541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.637556 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:43:24.637606 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.637610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.637667 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.637757 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:24.639874 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.640606 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.642242 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.642656 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.642747 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.646110 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:24.646114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:24.646117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:24.646146 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:24.649582 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.650363 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.650368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.650371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.650547 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.650916 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.650921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.650926 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.651306 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:24.651610 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.651917 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.652080 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1430,"thread_start_us":1323,"threads_started":1}
I20260926 07:43:24.654685 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.634871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:36313 (local address 127.11.9.126:46499)
0926 07:43:24.635093 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.635099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.636438 (+ 1339us) server_negotiation.cc:407] Connection header received
0926 07:43:24.639091 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.639094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.639143 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.639221 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:24.640843 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.641590 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.643705 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.644096 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.644181 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.647966 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:24.647974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:24.647983 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:24.648036 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:24.651442 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.652361 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.652369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.652373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.652509 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.652979 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.652986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.652990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.653374 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:24.653508 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.654261 (+ 753us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.654438 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 07:43:24.655870 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.635409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:36313)
0926 07:43:24.636225 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:24.636323 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:24.638773 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:24.639502 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:24.639515 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:24.640124 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:24.640656 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.640704 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.641777 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.641783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.643542 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.643572 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.644322 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.644329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.644652 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.645352 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:24.645385 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:24.647718 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:24.651576 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:24.651587 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:24.651605 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:24.652234 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:24.652617 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:24.652620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:24.652622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:24.652858 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:24.653510 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:24.653516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:24.653862 (+ 346us) client_negotiation.cc:769] Sending connection context
0926 07:43:24.654168 (+ 306us) client_negotiation.cc:240] Negotiation successful
0926 07:43:24.654739 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":351,"threads_started":1}
I20260926 07:43:24.657238 27805 heartbeater.cc:347] Connected to a master server at 127.11.9.126:46499
I20260926 07:43:24.657508 27805 heartbeater.cc:464] Registering TS with master...
I20260926 07:43:24.658152 27805 heartbeater.cc:511] Master 127.11.9.126:46499 requested a full tablet report, sending...
I20260926 07:43:24.659495 27357 ts_manager.cc:194] Registered new tserver with Master: 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:24.660291 27357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:36313
I20260926 07:43:24.667531 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:43:24.674292 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.668398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.11.9.126:46499)
0926 07:43:24.668549 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.668553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.669065 (+ 512us) server_negotiation.cc:407] Connection header received
0926 07:43:24.669282 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.669286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.669438 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.669560 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:43:24.670107 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.670847 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.672367 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.672832 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.672925 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.673481 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:43:24.673738 (+ 257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 07:43:24.674026 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32810: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 07:43:24.674424 27544 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32810: token expired
W20260926 07:43:24.674829 27268 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.9.126:46499)
I20260926 07:43:24.687043 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.675280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.11.9.126:46499)
0926 07:43:24.675419 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.675423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.675437 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:43:24.675545 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.675548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.675610 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.675683 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:24.676977 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.677701 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.678985 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.679353 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.679438 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.682122 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:24.682127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:24.682130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:24.682158 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:24.684768 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.685512 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.685516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.685518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.685627 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:24.685939 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:24.685942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:24.685944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:24.686221 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:24.686397 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.686680 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.686837 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 07:43:24.690578 27357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:43:24.698086 27357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:43:24.735392 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.710042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.11.9.65:45039)
0926 07:43:24.724754 (+ 14712us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.724762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.724799 (+ 37us) server_negotiation.cc:407] Connection header received
0926 07:43:24.724901 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.724909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.725180 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.728240 (+ 3060us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:24.728487 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.729862 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.732987 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.734169 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.734319 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.734549 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.734919 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.735077 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14541,"thread_start_us":17348,"threads_started":1}
I20260926 07:43:24.735512 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.709839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57190)
0926 07:43:24.720051 (+ 10212us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:24.720106 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:24.720331 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:24.727457 (+ 7126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:24.727462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:24.727488 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:24.728020 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.728034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.730317 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.730322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.732779 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.732797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.734411 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.734418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.734690 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.734728 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 07:43:24.734872 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 07:43:24.735065 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9868,"mutex_wait_us":52,"thread_start_us":13057,"threads_started":1}
I20260926 07:43:24.736331 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.709987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46587 (local address 127.0.0.1:42962)
0926 07:43:24.723164 (+ 13177us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:24.725351 (+ 2187us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:24.725542 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:24.726681 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:24.726689 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:24.726726 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:24.727137 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.727152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.731417 (+ 4265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.731425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.733261 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.733276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.735481 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.735487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.735675 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.735715 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 07:43:24.735818 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 07:43:24.735947 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13053,"mutex_wait_us":3,"thread_start_us":19661,"threads_started":1}
I20260926 07:43:24.737031 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.710442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.11.9.66:46587)
0926 07:43:24.722486 (+ 12044us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.722499 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.725436 (+ 2937us) server_negotiation.cc:407] Connection header received
0926 07:43:24.725734 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.725742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.725971 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.726137 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:24.729920 (+ 3783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.731233 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.733440 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.734992 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.735128 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.735320 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.736563 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.736735 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11823,"thread_start_us":12047,"threads_started":1}
I20260926 07:43:24.738196 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.712503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.0.0.1:53524)
0926 07:43:24.729477 (+ 16974us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:24.729496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:24.729605 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:24.730145 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:24.730149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:24.730174 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:24.730557 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.730571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.732857 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.732863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.735026 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:24.735042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.737405 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.737412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:24.737586 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.737621 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 07:43:24.737724 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 07:43:24.737849 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16842,"mutex_wait_us":5,"thread_start_us":7025,"threads_started":1}
I20260926 07:43:24.738245 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:24.713291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.11.9.67:38355)
0926 07:43:24.718044 (+ 4753us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:24.718057 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:24.729624 (+ 11567us) server_negotiation.cc:407] Connection header received
0926 07:43:24.729729 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:24.729736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:24.729944 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:24.730140 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:24.730750 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.732717 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.735199 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:24.737180 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:24.737328 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:24.737545 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:24.737765 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 07:43:24.737921 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4506,"thread_start_us":4302,"threads_started":1}
I20260926 07:43:24.739449 27477 tablet_service.cc:1467] Processing CreateTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 (DEFAULT_TABLE table=test-table [id=86ecb7df0e184628905395d147b97560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:24.741000 27477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9e35e242ba4637bf0bc67a6a8ebd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:24.741746 27740 tablet_service.cc:1467] Processing CreateTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 (DEFAULT_TABLE table=test-table [id=86ecb7df0e184628905395d147b97560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:24.743361 27740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9e35e242ba4637bf0bc67a6a8ebd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:24.744011 27609 tablet_service.cc:1467] Processing CreateTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 (DEFAULT_TABLE table=test-table [id=86ecb7df0e184628905395d147b97560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:43:24.744994 27609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9e35e242ba4637bf0bc67a6a8ebd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:43:24.756924 27822 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Bootstrap starting.
I20260926 07:43:24.758639 27823 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Bootstrap starting.
I20260926 07:43:24.759268 27542 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:24.761533 27821 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Bootstrap starting.
I20260926 07:43:24.761720 27822 tablet_bootstrap.cc:654] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:24.761792 27823 tablet_bootstrap.cc:654] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:24.763063 27822 log.cc:824] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:24.763069 27823 log.cc:824] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:24.764896 27821 tablet_bootstrap.cc:654] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Neither blocks nor log segments found. Creating new log.
I20260926 07:43:24.766211 27821 log.cc:824] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:24.766319 27823 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: No bootstrap required, opened a new log
I20260926 07:43:24.766605 27823 ts_tablet_manager.cc:1397] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 07:43:24.767068 27822 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: No bootstrap required, opened a new log
I20260926 07:43:24.767380 27822 ts_tablet_manager.cc:1397] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 07:43:24.771679 27821 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: No bootstrap required, opened a new log
I20260926 07:43:24.771919 27821 ts_tablet_manager.cc:1397] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260926 07:43:24.774417 27823 raft_consensus.cc:348] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.774844 27823 raft_consensus.cc:374] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:24.774955 27823 raft_consensus.cc:729] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a380b8d02243889c2bc526a00bff13, State: Initialized, Role: FOLLOWER
I20260926 07:43:24.775604 27823 consensus_queue.cc:260] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.776676 27805 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:24.777275 27823 ts_tablet_manager.cc:1428] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260926 07:43:24.777937 27822 raft_consensus.cc:348] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.778345 27822 raft_consensus.cc:374] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:24.778474 27822 raft_consensus.cc:729] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75dd0db81a440dd88fefca00a1dde68, State: Initialized, Role: FOLLOWER
I20260926 07:43:24.779045 27822 consensus_queue.cc:260] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.780761 27822 ts_tablet_manager.cc:1428] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 07:43:24.782761 27821 raft_consensus.cc:348] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.783098 27821 raft_consensus.cc:374] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:43:24.783223 27821 raft_consensus.cc:729] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e53040d37c467da3c924d1466df77c, State: Initialized, Role: FOLLOWER
I20260926 07:43:24.783802 27821 consensus_queue.cc:260] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:24.784797 27674 heartbeater.cc:503] Master 127.11.9.126:46499 was elected leader, sending a full tablet report...
I20260926 07:43:24.785421 27821 ts_tablet_manager.cc:1428] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
W20260926 07:43:24.883880 27806 tablet.cc:2369] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:43:24.924571 27675 tablet.cc:2369] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:43:24.973717 27543 tablet.cc:2369] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:43:25.099745 27829 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:25.100107 27829 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:25.101837 27829 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:25.108889 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.102280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:52189 (local address 127.11.9.67:38355)
0926 07:43:25.102508 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.102517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.103158 (+ 641us) server_negotiation.cc:407] Connection header received
0926 07:43:25.103333 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.103340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.103423 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.103515 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:25.104128 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.104898 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.107021 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.107852 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.107997 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.108113 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.108484 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.108616 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:43:25.108942 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.102246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:34829 (local address 127.11.9.65:45039)
0926 07:43:25.102464 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.102471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.103337 (+ 866us) server_negotiation.cc:407] Connection header received
0926 07:43:25.103507 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.103514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.103587 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.103688 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:43:25.104296 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.105092 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.107026 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.108010 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.108138 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.108239 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.108584 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.108709 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:43:25.109035 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.102089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.11.9.66:52189)
0926 07:43:25.103052 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.103067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:25.103188 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:25.103544 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:25.103548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:25.103572 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:25.103974 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:25.103985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.105032 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.105039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:25.106876 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:25.106888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.108097 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.108105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:25.108305 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.108332 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 07:43:25.108416 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 07:43:25.108581 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":42,"thread_start_us":566,"threads_started":1}
I20260926 07:43:25.109064 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.102346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.11.9.66:34829)
0926 07:43:25.103201 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.103217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:25.103359 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:25.103718 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:25.103721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:25.103740 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:43:25.104117 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:25.104129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.105268 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.105272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:25.106879 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:25.106891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.108252 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.108256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:25.108389 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.108449 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 07:43:25.108532 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 07:43:25.108647 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":21,"thread_start_us":460,"threads_started":1}
I20260926 07:43:25.110097 27760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9e35e242ba4637bf0bc67a6a8ebd85" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a380b8d02243889c2bc526a00bff13" is_pre_election: true
I20260926 07:43:25.110227 27497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9e35e242ba4637bf0bc67a6a8ebd85" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75dd0db81a440dd88fefca00a1dde68" is_pre_election: true
I20260926 07:43:25.110607 27760 raft_consensus.cc:2454] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 0.
I20260926 07:43:25.110750 27497 raft_consensus.cc:2454] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 0.
I20260926 07:43:25.111235 27562 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13; no voters:
I20260926 07:43:25.111634 27829 raft_consensus.cc:2781] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:43:25.111781 27829 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:43:25.111907 27829 raft_consensus.cc:3037] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:25.114104 27829 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:25.114759 27829 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 election: Requested vote from peers c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
I20260926 07:43:25.115131 27497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9e35e242ba4637bf0bc67a6a8ebd85" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75dd0db81a440dd88fefca00a1dde68"
I20260926 07:43:25.115363 27497 raft_consensus.cc:3037] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:25.115298 27760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9e35e242ba4637bf0bc67a6a8ebd85" candidate_uuid: "22e53040d37c467da3c924d1466df77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a380b8d02243889c2bc526a00bff13"
I20260926 07:43:25.115537 27760 raft_consensus.cc:3037] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:43:25.117834 27497 raft_consensus.cc:2454] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 1.
I20260926 07:43:25.118139 27760 raft_consensus.cc:2454] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e53040d37c467da3c924d1466df77c in term 1.
I20260926 07:43:25.118243 27565 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e53040d37c467da3c924d1466df77c, c75dd0db81a440dd88fefca00a1dde68; no voters:
I20260926 07:43:25.118547 27829 raft_consensus.cc:2781] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:43:25.118963 27829 raft_consensus.cc:686] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 1 LEADER]: Becoming Leader. State: Replica: 22e53040d37c467da3c924d1466df77c, State: Running, Role: LEADER
I20260926 07:43:25.119405 27829 consensus_queue.cc:237] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:25.123994 27356 catalog_manager.cc:5495] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c reported cstate change: term changed from 0 to 1, leader changed from <none> to 22e53040d37c467da3c924d1466df77c (127.11.9.66). New cstate: current_term: 1 leader_uuid: "22e53040d37c467da3c924d1466df77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } health_report { overall_health: UNKNOWN } } }
I20260926 07:43:25.188575 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.183511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42974 (local address 127.11.9.66:46587)
0926 07:43:25.183719 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.183723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.183797 (+ 74us) server_negotiation.cc:407] Connection header received
0926 07:43:25.183971 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.183975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.184050 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.184173 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:43:25.184486 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.185235 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.186526 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.186952 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.187043 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.187505 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:43:25.187974 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:43:25.188141 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.188195 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.188274 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 07:43:25.196436 27760 raft_consensus.cc:1261] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Refusing update from remote peer 22e53040d37c467da3c924d1466df77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:43:25.196434 27497 raft_consensus.cc:1261] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Refusing update from remote peer 22e53040d37c467da3c924d1466df77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:43:25.197422 27832 consensus_queue.cc:1035] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:43:25.197757 27829 consensus_queue.cc:1035] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:43:25.208421 27834 mvcc.cc:205] Tried to move back new op lower bound from 7333513646875164672 to 7333513646572777472. Current Snapshot: MvccSnapshot[applied={T|T < 7333513646875164672}]
I20260926 07:43:25.210561 27836 mvcc.cc:205] Tried to move back new op lower bound from 7333513646875164672 to 7333513646572777472. Current Snapshot: MvccSnapshot[applied={T|T < 7333513646875164672}]
I20260926 07:43:25.222411 27357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:43:25.222702 27357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:43:25.226334 27357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Sending DeleteTablet for 3 replicas of tablet fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:25.227418 27740 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:53524
I20260926 07:43:25.227421 27609 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:42962
I20260926 07:43:25.227416 27477 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9e35e242ba4637bf0bc67a6a8ebd85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:43:25 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:57190
I20260926 07:43:25.229851 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27394
I20260926 07:43:25.243960 27844 tablet_replica.cc:330] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: stopping tablet replica
I20260926 07:43:25.244899 27844 raft_consensus.cc:2229] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:43:25.245519 27844 raft_consensus.cc:2258] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:25.246342 27846 tablet_replica.cc:330] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: stopping tablet replica
I20260926 07:43:25.246798 27846 raft_consensus.cc:2229] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:43:25.247287 27846 raft_consensus.cc:2258] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:25.249833 27844 ts_tablet_manager.cc:1905] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:43:25.257126 27844 ts_tablet_manager.cc:1918] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:43:25.257570 27845 tablet_replica.cc:330] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: stopping tablet replica
I20260926 07:43:25.257961 27845 raft_consensus.cc:2229] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:43:25.258580 27845 raft_consensus.cc:2258] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:43:25.258747 27846 ts_tablet_manager.cc:1905] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:43:25.259718 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.231098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.11.9.65:45039)
0926 07:43:25.231331 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.231341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.237649 (+ 6308us) server_negotiation.cc:407] Connection header received
0926 07:43:25.238836 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.238846 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.238967 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.239058 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:25.244493 (+ 5435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.245693 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.247596 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.248170 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.248275 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.251873 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:25.251886 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:25.251903 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:25.251940 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:25.255664 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.256622 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.256631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.256635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.256802 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.257322 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.257329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.257332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.257822 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:25.258102 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.258684 (+ 582us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.258904 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:43:25.260308 27845 ts_tablet_manager.cc:1905] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:43:25.261211 27537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:25.263942 27844 log.cc:1196] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:25.264649 27844 ts_tablet_manager.cc:1939] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 87a380b8d02243889c2bc526a00bff13: Deleting consensus metadata
I20260926 07:43:25.266533 27341 catalog_manager.cc:4841] TS 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): tablet fe9e35e242ba4637bf0bc67a6a8ebd85 (table test-table [id=86ecb7df0e184628905395d147b97560]) successfully deleted
I20260926 07:43:25.270929 27845 ts_tablet_manager.cc:1918] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:43:25.271168 27845 log.cc:1196] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:25.271891 27845 ts_tablet_manager.cc:1939] T fe9e35e242ba4637bf0bc67a6a8ebd85 P 22e53040d37c467da3c924d1466df77c: Deleting consensus metadata
I20260926 07:43:25.273892 27342 catalog_manager.cc:4841] TS 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): tablet fe9e35e242ba4637bf0bc67a6a8ebd85 (table test-table [id=86ecb7df0e184628905395d147b97560]) successfully deleted
I20260926 07:43:25.354243 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27394
W20260926 07:43:25.378849 27344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:45039 (error 108)
I20260926 07:43:25.380128 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27546
I20260926 07:43:25.381659 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.380735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57202)
0926 07:43:25.381249 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.381386 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":216,"threads_started":1}
W20260926 07:43:25.382501 27344 catalog_manager.cc:4584] TS c75dd0db81a440dd88fefca00a1dde68 (127.11.9.65:45039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe9e35e242ba4637bf0bc67a6a8ebd85: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
I20260926 07:43:25.395673 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.382511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.11.9.66:46587)
0926 07:43:25.382691 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.382697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.382715 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:43:25.382795 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.382800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.382882 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.383010 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:25.384442 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.385448 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.386712 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.387132 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.387279 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.389822 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:25.389834 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:25.389848 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:25.389886 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:25.392841 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.393561 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.393576 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.393578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.393711 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.394081 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.394087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.394090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.394434 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:25.394799 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.395061 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.395275 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:43:25.396399 27669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:25.438750 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.438132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57216)
0926 07:43:25.438379 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.438525 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 07:43:25.439949 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.439433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57220)
0926 07:43:25.439642 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.439763 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 07:43:25.500530 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.500010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57230)
0926 07:43:25.500290 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.500382 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260926 07:43:25.501631 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.501276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57242)
0926 07:43:25.501410 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.501489 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 07:43:25.512315 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27546
I20260926 07:43:25.530557 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27677
I20260926 07:43:25.544096 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.531491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.11.9.67:38355)
0926 07:43:25.531655 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:25.531660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:25.531677 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:43:25.531819 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:25.531824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:25.531904 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:25.531993 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:25.533496 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.534238 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.535495 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:25.535888 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:25.536023 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:25.538560 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:25.538572 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:25.538587 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:25.538616 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:25.541413 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.542102 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.542110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.542113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.542261 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:25.542616 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:25.542640 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:25.542643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:25.542982 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:25.543252 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:25.543566 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 07:43:25.543775 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:43:25.544768 27800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:25.582135 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.581668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57254)
0926 07:43:25.581891 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.581982 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 07:43:25.583292 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:25.582882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:45039 (local address 127.0.0.1:57262)
0926 07:43:25.583077 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:25.583146 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 07:43:25.632725 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27677
I20260926 07:43:25.652747 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27324
I20260926 07:43:25.653877 27387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:43:25.768023 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27324
I20260926 07:43:45.789985 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--webserver_interface=127.11.9.126
--webserver_port=33745
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:46499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:43:46.121047 27854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:46.121443 27854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:46.121605 27854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:46.128769 27854 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 07:43:46.128894 27854 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 07:43:46.128986 27854 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 07:43:46.129084 27854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:43:46.129205 27854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:46.129324 27854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:46.129413 27854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:43:46.129490 27854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:43:46.139088 27854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:46499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=33745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:46.140291 27854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:46.141620 27854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:46.153707 27862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:46.153728 27859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:46.153955 27854 server_base.cc:1034] running on GCE node
W20260926 07:43:46.153707 27860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:46.155043 27854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:46.156365 27854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:46.157599 27854 hybrid_clock.cc:648] HybridClock initialized: now 1790408626157526 us; error 54 us; skew 500 ppm
I20260926 07:43:46.157915 27854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:46 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:46.161011 27854 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:43:46.162431 27854 webserver.cc:460] Webserver started at http://127.11.9.126:33745/ using document root <none> and password file <none>
I20260926 07:43:46.163087 27854 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:46.163170 27854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:46.168260 27854 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 07:43:46.170974 27869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:46.172279 27854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:43:46.172416 27854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "f73e5bea656743159c18152df1e340ee"
format_stamp: "Formatted at 2026-09-26 07:42:59 on dist-test-slave-x492"
server_key: "a049d55d3a18ad022ecceb02cee12734"
server_key_iv: "b396b8bd3cefb5b81042d068b83b048e"
server_key_version: "encryptionkey@0"
I20260926 07:43:46.173444 27854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:46.189572 27854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:46.193389 27854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:46.193614 27854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:46.212119 27854 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:46499
I20260926 07:43:46.212164 27920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:46499 every 8 connection(s)
I20260926 07:43:46.213616 27854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:43:46.214112 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27854
I20260926 07:43:46.215219 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:45039
--local_ip_for_outbound_sockets=127.11.9.65
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=38275
--webserver_interface=127.11.9.65
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:43:46.223577 27921 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:43:46.226886 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap starting.
I20260926 07:43:46.247262 27921 log.cc:824] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:46.262832 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:43:46.263388 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Bootstrap complete.
I20260926 07:43:46.272271 27921 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:46.273149 27921 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73e5bea656743159c18152df1e340ee, State: Initialized, Role: FOLLOWER
I20260926 07:43:46.273743 27921 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:46.273990 27921 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:43:46.274132 27921 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:43:46.274326 27921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 2 FOLLOWER]: Advancing to term 3
I20260926 07:43:46.278328 27921 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:46.278805 27921 leader_election.cc:302] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73e5bea656743159c18152df1e340ee; no voters:
I20260926 07:43:46.279493 27921 leader_election.cc:288] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 07:43:46.279633 27925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 3 FOLLOWER]: Leader election won for term 3
I20260926 07:43:46.281792 27921 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:43:46.286674 27925 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [term 3 LEADER]: Becoming Leader. State: Replica: f73e5bea656743159c18152df1e340ee, State: Running, Role: LEADER
I20260926 07:43:46.287319 27925 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } }
I20260926 07:43:46.289987 27926 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:43:46.290393 27926 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:43:46.290990 27927 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73e5bea656743159c18152df1e340ee. Latest consensus state: current_term: 3 leader_uuid: "f73e5bea656743159c18152df1e340ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73e5bea656743159c18152df1e340ee" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 46499 } } }
I20260926 07:43:46.291324 27927 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee [sys.catalog]: This master's current role is: LEADER
I20260926 07:43:46.293478 27936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:43:46.310411 27936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:43:46.312433 27936 catalog_manager.cc:1259] Loaded cluster ID: 85519c9e83d34620808867268525f5b7
I20260926 07:43:46.312629 27936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:43:46.318143 27936 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:43:46.320276 27936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Loaded TSK: 0
I20260926 07:43:46.320533 27936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f73e5bea656743159c18152df1e340ee: Loaded TSK: 1
F20260926 07:43:46.322057 27936 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f53a05af9ad google::LogMessage::Fail() at ??:0
@ 0x7f53a05b288a google::LogMessage::SendToLog() at ??:0
@ 0x7f53a05af4de google::LogMessage::Flush() at ??:0
@ 0x7f53a05b0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f53a43137d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f53a430273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f53b5fde1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f53b5fdd783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f53b60692a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f53b60690c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f53b60b35e9 std::function<>::operator()() at ??:0
@ 0x7f53b5fe1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f53b5fdfc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f53b6067610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f53b6067439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f53a31fb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f53a31fe91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f53a31fe799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f53a31d9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f53a64b36db start_thread at ??:0
@ 0x7f539f44961f clone at ??:0
W20260926 07:43:46.606209 27923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:46.606491 27923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:46.606654 27923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:46.614080 27923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:46.614209 27923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:46.614398 27923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:43:46.624210 27923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:45039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=38275
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:46.625491 27923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:46.626789 27923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:46.639648 27969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:46.639739 27970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:46.639767 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:46.641037 27923 server_base.cc:1034] running on GCE node
I20260926 07:43:46.641711 27923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:46.642688 27923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:46.643919 27923 hybrid_clock.cc:648] HybridClock initialized: now 1790408626643847 us; error 65 us; skew 500 ppm
I20260926 07:43:46.644284 27923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:46 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:46.647763 27923 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:43:46.649411 27923 webserver.cc:460] Webserver started at http://127.11.9.65:38275/ using document root <none> and password file <none>
I20260926 07:43:46.650228 27923 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:46.650388 27923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:46.656186 27923 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260926 07:43:46.659286 27979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:46.660671 27923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:43:46.660853 27923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "c75dd0db81a440dd88fefca00a1dde68"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "aacf83c77306f208534e30ab367b4a7d"
server_key_iv: "138e600d60cd2309611c299061f61824"
server_key_version: "encryptionkey@0"
I20260926 07:43:46.661898 27923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:46.675885 27923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:46.679656 27923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:46.679930 27923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:46.680905 27923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:46.682655 27986 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 07:43:46.688071 27923 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 07:43:46.688226 27923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260926 07:43:46.688377 27923 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 07:43:46.691593 27923 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 07:43:46.691685 27923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260926 07:43:46.692095 27986 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Bootstrap starting.
I20260926 07:43:46.700925 27986 log.cc:824] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Log is configured to *not* fsync() on all Append() calls
I20260926 07:43:46.717345 27986 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:43:46.717999 27986 tablet_bootstrap.cc:492] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Bootstrap complete.
I20260926 07:43:46.718406 27986 ts_tablet_manager.cc:1397] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Time spent bootstrapping tablet: real 0.027s user 0.024s sys 0.000s
I20260926 07:43:46.728155 27986 raft_consensus.cc:348] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:46.729835 27986 raft_consensus.cc:729] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75dd0db81a440dd88fefca00a1dde68, State: Initialized, Role: FOLLOWER
I20260926 07:43:46.730576 27986 consensus_queue.cc:260] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:46.732096 27986 ts_tablet_manager.cc:1428] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68: Time spent starting tablet: real 0.014s user 0.002s sys 0.012s
I20260926 07:43:46.737213 27923 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:45039
I20260926 07:43:46.737299 28094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:45039 every 8 connection(s)
I20260926 07:43:46.738430 27923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:43:46.742249 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 27923
I20260926 07:43:46.742941 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:46587
--local_ip_for_outbound_sockets=127.11.9.66
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=44809
--webserver_interface=127.11.9.66
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:43:46.742964 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:46.741040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:38383)
0926 07:43:46.741826 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:46.742034 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":359,"threads_started":1}
W20260926 07:43:46.743675 28095 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:46499 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:46499: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
I20260926 07:43:46.744638 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:46.744214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:34359)
0926 07:43:46.744401 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:46.744494 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 07:43:46.745756 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:46.745314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:48687)
0926 07:43:46.745495 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:46.745590 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 07:43:46.746047 28095 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:43:47.092134 28098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:47.092415 28098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:47.092586 28098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:47.099881 28098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:47.099983 28098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:47.100172 28098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:43:47.109800 28098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:46587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=44809
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:47.111080 28098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:47.112349 28098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:47.125602 28106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:47.125571 28104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:47.125643 28103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:47.130551 28098 server_base.cc:1034] running on GCE node
I20260926 07:43:47.131168 28098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:47.132061 28098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:47.133287 28098 hybrid_clock.cc:648] HybridClock initialized: now 1790408627133218 us; error 55 us; skew 500 ppm
I20260926 07:43:47.133589 28098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:47 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:47.136731 28098 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:43:47.138293 28098 webserver.cc:460] Webserver started at http://127.11.9.66:44809/ using document root <none> and password file <none>
I20260926 07:43:47.138998 28098 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:47.139153 28098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:47.144367 28098 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 07:43:47.147104 28113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.148432 28098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:43:47.148635 28098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "22e53040d37c467da3c924d1466df77c"
format_stamp: "Formatted at 2026-09-26 07:43:00 on dist-test-slave-x492"
server_key: "8efa376a0117b82678b1b498159c458a"
server_key_iv: "ecf7de0719a9cb5de84def486bca0ec5"
server_key_version: "encryptionkey@0"
I20260926 07:43:47.149729 28098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:47.161665 28098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:47.165516 28098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:47.165784 28098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:47.166934 28098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:47.168213 28098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:47.168334 28098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.168462 28098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:47.168555 28098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.201864 28098 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:46587
I20260926 07:43:47.201957 28225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:46587 every 8 connection(s)
I20260926 07:43:47.203449 28098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:43:47.207603 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28098
I20260926 07:43:47.208290 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:38355
--local_ip_for_outbound_sockets=127.11.9.67
--tserver_master_addrs=127.11.9.126:46499
--webserver_port=42037
--webserver_interface=127.11.9.67
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:43:47.208653 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.206623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:57619)
0926 07:43:47.207382 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.207621 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":282,"threads_started":1}
W20260926 07:43:47.209455 28226 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:46499 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:46499: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
I20260926 07:43:47.210314 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.209978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:34387)
0926 07:43:47.210135 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.210213 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 07:43:47.211210 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.210884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:49497)
0926 07:43:47.211027 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.211111 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 07:43:47.211519 28226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 07:43:47.530674 28229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:43:47.531051 28229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:43:47.531216 28229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:43:47.538189 28229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:43:47.538280 28229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:43:47.538384 28229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:43:47.547569 28229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:38355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=42037
--tserver_master_addrs=127.11.9.126:46499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:43:47.548723 28229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:43:47.550072 28229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:43:47.563010 28234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:47.562978 28235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:43:47.562927 28237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:43:47.563865 28229 server_base.cc:1034] running on GCE node
I20260926 07:43:47.564380 28229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:43:47.565235 28229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:43:47.566440 28229 hybrid_clock.cc:648] HybridClock initialized: now 1790408627566367 us; error 65 us; skew 500 ppm
I20260926 07:43:47.566807 28229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:43:47 dist-test-slave-x492 krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:43:47.569907 28229 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:43:47.571352 28229 webserver.cc:460] Webserver started at http://127.11.9.67:42037/ using document root <none> and password file <none>
I20260926 07:43:47.572019 28229 fs_manager.cc:359] Metadata directory not provided
I20260926 07:43:47.572134 28229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:43:47.577340 28229 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 07:43:47.579844 28244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.581130 28229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:43:47.581354 28229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "87a380b8d02243889c2bc526a00bff13"
format_stamp: "Formatted at 2026-09-26 07:43:01 on dist-test-slave-x492"
server_key: "cfbd767d3fe38ec22b289724917f2332"
server_key_iv: "59cc297c4f9f0ce6541d25cafb2380e2"
server_key_version: "encryptionkey@0"
I20260926 07:43:47.582314 28229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:43:47.617470 28229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:43:47.621879 28229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:43:47.622167 28229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:43:47.623274 28229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:43:47.624552 28229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:43:47.624642 28229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.624797 28229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:43:47.624898 28229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:43:47.663802 28229 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38355
I20260926 07:43:47.663890 28356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38355 every 8 connection(s)
I20260926 07:43:47.665311 28229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:43:47.670184 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.668280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:37355)
0926 07:43:47.668969 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.669253 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":261,"threads_started":1}
W20260926 07:43:47.670967 28357 heartbeater.cc:644] Failed to heartbeat to 127.11.9.126:46499 (0 consecutive failures): Network error: Failed to ping master at 127.11.9.126:46499: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
I20260926 07:43:47.671236 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28229
I20260926 07:43:47.671850 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.671472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:57107)
0926 07:43:47.671641 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.671723 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 07:43:47.673345 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.672952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:38139)
0926 07:43:47.673102 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.673219 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 07:43:47.673584 28357 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 07:43:47.747257 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:47.746524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:36477)
0926 07:43:47.746970 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:47.747061 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":223,"threads_started":1}
I20260926 07:43:48.212838 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.212069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:33725)
0926 07:43:48.212538 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:48.212616 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":253,"threads_started":1}
I20260926 07:43:48.349567 28363 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:48.349916 28363 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:48.351588 28363 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
Sep 26 07:43:48 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
Sep 26 07:43:48 dist-test-slave-x492 krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:43:48.392671 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.351876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46587 (local address 127.11.9.65:50217)
0926 07:43:48.363755 (+ 11879us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:48.363800 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:48.364496 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:48.368101 (+ 3605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:48.368113 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:48.369330 (+ 1217us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:48.370480 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:48.370507 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.372940 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.372947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:48.375755 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:48.375772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.377269 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.377277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:48.377800 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:48.381702 (+ 3902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:48.381746 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:48.384873 (+ 3127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:48.388533 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:48.388543 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:48.388559 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:48.389222 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:48.389678 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:48.389683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:48.389692 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:48.389952 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:48.390912 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:48.390918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:48.391385 (+ 467us) client_negotiation.cc:769] Sending connection context
0926 07:43:48.391719 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 07:43:48.392206 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11754,"mutex_wait_us":330,"spinlock_wait_cycles":67072,"thread_start_us":12061,"threads_started":1}
I20260926 07:43:48.393010 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.352290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:50217 (local address 127.11.9.66:46587)
0926 07:43:48.364663 (+ 12373us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:48.364677 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:48.364712 (+ 35us) server_negotiation.cc:407] Connection header received
0926 07:43:48.367038 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:48.367047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:48.367545 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:48.367930 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:48.370707 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.372682 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.375937 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.376977 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.377194 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:48.385272 (+ 8078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:48.385286 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:48.385308 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:48.385374 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:48.388380 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:48.389369 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:48.389374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:48.389380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:48.389546 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:48.390086 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:48.390094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:48.390100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:48.390763 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:48.390961 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:48.392203 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0926 07:43:48.392632 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12197,"thread_start_us":12047,"threads_started":1}
I20260926 07:43:48.393036 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.351897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38355 (local address 127.11.9.65:57895)
0926 07:43:48.364117 (+ 12220us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:48.364132 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:43:48.364422 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:43:48.368107 (+ 3685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:43:48.368113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:43:48.369330 (+ 1217us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:43:48.370480 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:48.370508 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.372943 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.372948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:48.375754 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:43:48.375772 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.377280 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.377284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:43:48.377798 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:48.381702 (+ 3904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:43:48.381748 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:43:48.385165 (+ 3417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:43:48.389850 (+ 4685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:48.389853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:48.389856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:48.390309 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:48.390769 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:43:48.390773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:43:48.390777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:43:48.390952 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:43:48.392047 (+ 1095us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:43:48.392052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:43:48.392215 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 07:43:48.392348 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 07:43:48.392606 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12103,"mutex_wait_us":541,"spinlock_wait_cycles":13440,"thread_start_us":12442,"threads_started":1}
I20260926 07:43:48.393532 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.352305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:57895 (local address 127.11.9.67:38355)
0926 07:43:48.363880 (+ 11575us) server_negotiation.cc:206] Beginning negotiation
0926 07:43:48.363901 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 07:43:48.364606 (+ 705us) server_negotiation.cc:407] Connection header received
0926 07:43:48.367032 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:43:48.367043 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:43:48.367545 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:43:48.367931 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:43:48.370713 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.372681 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.375937 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:43:48.376977 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:43:48.377208 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:43:48.385458 (+ 8250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:43:48.385466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:43:48.385511 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:43:48.385574 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:43:48.389702 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:48.390473 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:48.390479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:48.390483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:48.390641 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:43:48.391089 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:43:48.391094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:43:48.391097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:43:48.391814 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:43:48.391986 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 07:43:48.392649 (+ 663us) server_negotiation.cc:299] Negotiation successful
0926 07:43:48.393058 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11348,"thread_start_us":11204,"threads_started":1}
W20260926 07:43:48.396306 27981 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
W20260926 07:43:48.396579 27980 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:48.396749 27980 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:43:48.397217 28363 raft_consensus.cc:2726] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:43:48.675074 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.674072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:55125)
0926 07:43:48.674601 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:48.674794 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":273,"threads_started":1}
I20260926 07:43:48.748821 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:48.748298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:40711)
0926 07:43:48.748511 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:48.748654 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 07:43:49.214429 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:49.213761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:59985)
0926 07:43:49.214187 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:49.214264 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":234,"threads_started":1}
I20260926 07:43:49.676573 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:49.675934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:47877)
0926 07:43:49.676334 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:49.676412 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":214,"threads_started":1}
I20260926 07:43:49.750559 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:49.749891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:48339)
0926 07:43:49.750303 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:49.750382 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":209,"threads_started":1}
I20260926 07:43:50.063767 28372 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:50.063975 28372 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:50.064621 28372 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
W20260926 07:43:50.065788 27981 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
W20260926 07:43:50.066035 27980 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:50.066179 27980 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:43:50.066529 28372 raft_consensus.cc:2726] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:43:50.216069 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:50.215334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:38827)
0926 07:43:50.215828 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:50.215912 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":278,"threads_started":1}
I20260926 07:43:50.678274 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:50.677579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:57939)
0926 07:43:50.678019 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:50.678108 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":230,"threads_started":1}
I20260926 07:43:50.752171 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:50.751416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:58533)
0926 07:43:50.751905 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:50.751992 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":280,"threads_started":1}
I20260926 07:43:51.217804 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:51.217001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:36013)
0926 07:43:51.217537 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:51.217624 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":296,"threads_started":1}
I20260926 07:43:51.679797 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:51.679120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:42235)
0926 07:43:51.679539 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:51.679615 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":209,"threads_started":1}
I20260926 07:43:51.754067 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:51.753285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:57181)
0926 07:43:51.753778 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:51.753878 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":235,"threads_started":1}
I20260926 07:43:52.219483 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:52.218775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:58515)
0926 07:43:52.219215 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:52.219301 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260926 07:43:52.681402 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:52.680658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:59389)
0926 07:43:52.681092 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:52.681224 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":227,"threads_started":1}
I20260926 07:43:52.755872 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:52.754900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:55257)
0926 07:43:52.755404 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:52.755509 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":286,"threads_started":1}
I20260926 07:43:52.862175 28382 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:52.862376 28382 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:52.863030 28382 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
W20260926 07:43:52.863974 27980 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
W20260926 07:43:52.864176 27981 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:52.864323 27981 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:43:52.864605 28382 raft_consensus.cc:2726] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:43:53.221284 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:53.220415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:59805)
0926 07:43:53.220940 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:53.221022 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":306,"threads_started":1}
I20260926 07:43:53.682955 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:53.682250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:41409)
0926 07:43:53.682704 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:53.682783 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260926 07:43:53.757573 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:53.756787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:38445)
0926 07:43:53.757294 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:53.757388 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260926 07:43:54.223270 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:54.222322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:39229)
0926 07:43:54.222967 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:54.223070 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":306,"threads_started":1}
I20260926 07:43:54.684768 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:54.683929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:42581)
0926 07:43:54.684450 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:54.684564 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":229,"threads_started":1}
I20260926 07:43:54.759136 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:54.758415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:33839)
0926 07:43:54.758885 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:54.758975 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":262,"threads_started":1}
I20260926 07:43:55.224830 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:55.224162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:53501)
0926 07:43:55.224575 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:55.224661 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":228,"threads_started":1}
I20260926 07:43:55.686466 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:55.685736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:46745)
0926 07:43:55.686159 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:55.686235 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
I20260926 07:43:55.760712 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:55.760001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:43329)
0926 07:43:55.760454 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:55.760532 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":233,"threads_started":1}
I20260926 07:43:55.932416 28392 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:43:55.932716 28392 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:43:55.933459 28392 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
W20260926 07:43:55.934417 27980 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
W20260926 07:43:55.934651 27981 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:43:55.934754 27981 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:43:55.934981 28392 raft_consensus.cc:2726] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:43:56.226511 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:56.225790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:43793)
0926 07:43:56.226272 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:56.226358 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":222,"threads_started":1}
I20260926 07:43:56.687991 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:56.687307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:58559)
0926 07:43:56.687761 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:56.687843 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":225,"threads_started":1}
I20260926 07:43:56.762428 28395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:56.761753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:51529)
0926 07:43:56.762189 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:56.762268 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":229,"threads_started":1}
I20260926 07:43:57.228147 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:57.227442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:56045)
0926 07:43:57.227874 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:57.227977 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260926 07:43:57.689513 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:57.688812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:45439)
0926 07:43:57.689260 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:57.689338 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":210,"threads_started":1}
I20260926 07:43:57.764057 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:57.763285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:45463)
0926 07:43:57.763748 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:57.763870 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":241,"threads_started":1}
I20260926 07:43:58.229892 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:58.229057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:43535)
0926 07:43:58.229629 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:58.229730 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":329,"threads_started":1}
I20260926 07:43:58.691075 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:58.690374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:40597)
0926 07:43:58.690805 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:58.690917 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":221,"threads_started":1}
I20260926 07:43:58.765729 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:58.764932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:45847)
0926 07:43:58.765487 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:58.765566 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":314,"threads_started":1}
I20260926 07:43:59.231581 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:59.230912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:37265)
0926 07:43:59.231321 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:59.231399 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":221,"threads_started":1}
I20260926 07:43:59.692773 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:59.692027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:54573)
0926 07:43:59.692504 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:59.692587 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260926 07:43:59.767272 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:43:59.766537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:59171)
0926 07:43:59.767007 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 07:43:59.767095 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":246,"threads_started":1}
I20260926 07:44:00.233237 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:00.232481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:60231)
0926 07:44:00.232913 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:00.233003 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260926 07:44:00.458390 28406 raft_consensus.cc:482] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:00.458612 28406 raft_consensus.cc:504] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75dd0db81a440dd88fefca00a1dde68" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 45039 } } peers { permanent_uuid: "22e53040d37c467da3c924d1466df77c" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46587 } } peers { permanent_uuid: "87a380b8d02243889c2bc526a00bff13" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38355 } }
I20260926 07:44:00.459306 28406 leader_election.cc:288] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587), 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355)
W20260926 07:44:00.460201 27981 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22e53040d37c467da3c924d1466df77c (127.11.9.66:46587): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
W20260926 07:44:00.460392 27980 leader_election.cc:341] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87a380b8d02243889c2bc526a00bff13 (127.11.9.67:38355): Not found: Tablet not found: fe9e35e242ba4637bf0bc67a6a8ebd85
I20260926 07:44:00.460530 27980 leader_election.cc:302] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75dd0db81a440dd88fefca00a1dde68; no voters: 22e53040d37c467da3c924d1466df77c, 87a380b8d02243889c2bc526a00bff13
I20260926 07:44:00.460773 28406 raft_consensus.cc:2726] T fe9e35e242ba4637bf0bc67a6a8ebd85 P c75dd0db81a440dd88fefca00a1dde68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:44:00.694595 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:00.693729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:56171)
0926 07:44:00.694252 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:00.694366 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":298,"threads_started":1}
I20260926 07:44:00.769028 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:00.768185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:35079)
0926 07:44:00.768636 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:00.768753 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":225,"threads_started":1}
I20260926 07:44:01.234872 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:01.234147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:49019)
0926 07:44:01.234602 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:01.234682 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":233,"threads_started":1}
I20260926 07:44:01.696485 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:01.695475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:44093)
0926 07:44:01.696206 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:01.696291 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":512,"threads_started":1}
I20260926 07:44:01.770800 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:01.770131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:37321)
0926 07:44:01.770541 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:01.770618 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":213,"threads_started":1}
I20260926 07:44:02.236651 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.235935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.66:60305)
0926 07:44:02.236379 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:02.236466 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260926 07:44:02.685067 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 27923
I20260926 07:44:02.698287 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.697565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.67:53323)
0926 07:44:02.698039 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:02.698123 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":252,"threads_started":1}
I20260926 07:44:02.705504 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.686216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.11.9.65:45039)
0926 07:44:02.686711 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:02.686719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:02.686748 (+ 29us) server_negotiation.cc:407] Connection header received
0926 07:44:02.686836 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:02.686849 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:02.687155 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:02.687343 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:02.688630 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.690065 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.691237 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.691752 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.691848 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:02.699838 (+ 7990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:02.699850 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:02.699865 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:02.699904 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:02.702626 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.703318 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.703326 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.703329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.703472 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.703862 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.703867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.703871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.704242 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:02.704562 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:02.704858 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 07:44:02.705140 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260926 07:44:02.706737 28090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:02.797824 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.796718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:46499 (local address 127.11.9.65:56661)
0926 07:44:02.797359 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:02.797522 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.126:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":321,"threads_started":1}
I20260926 07:44:02.801350 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 27923
I20260926 07:44:02.818559 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28098
I20260926 07:44:02.831346 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.819325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.11.9.66:46587)
0926 07:44:02.819852 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:02.819856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:02.819873 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:44:02.819932 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:02.819935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:02.819994 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:02.820085 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:02.821496 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.822306 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.823477 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.823909 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.824002 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:02.826536 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:02.826540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:02.826544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:02.826576 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:02.829047 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.829688 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.829692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.829694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.829825 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.830131 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.830134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.830136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.830385 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:02.830574 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:02.830791 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 07:44:02.830960 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":284,"threads_started":1}
I20260926 07:44:02.832278 28221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:02.919293 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28098
I20260926 07:44:02.936856 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28229
I20260926 07:44:02.950598 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:02.937678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.11.9.67:38355)
0926 07:44:02.938253 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:02.938258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:02.938273 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:44:02.938327 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:02.938331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:02.938390 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:02.938515 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:02.939944 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.940714 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.942091 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:02.942533 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:02.942673 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:02.945282 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:02.945288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:02.945292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:02.945324 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:02.947752 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.948427 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.948434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.948437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.948590 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:02.948971 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:02.948977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:02.948986 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:02.949304 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:02.949613 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:02.949889 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 07:44:02.950131 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":284,"threads_started":1}
I20260926 07:44:02.951332 28352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:03.034446 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28229
2026-09-26T07:44:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64867 ms)
[----------] 30 tests from SecurityITest (291459 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:03 dist-test-slave-x492 krb5kdc[28423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:03 dist-test-slave-x492 krb5kdc[28423](info): set up 2 sockets
Sep 26 07:44:03 dist-test-slave-x492 krb5kdc[28423](info): commencing operation
krb5kdc: starting...
W20260926 07:44:04.141386 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:04 dist-test-slave-x492 krb5kdc[28423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:04.371034 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:32887
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:32887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:04.695802 28436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:04.696080 28436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:04.696208 28436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:04.703353 28436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:04.703450 28436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:04.703496 28436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:04.703553 28436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:04.703598 28436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:04.713138 28436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:32887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:32887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:04.714433 28436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:04.715734 28436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:04.727643 28441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:04.727582 28442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:04.727610 28444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:04.727960 28436 server_base.cc:1034] running on GCE node
I20260926 07:44:04.728947 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:04.730296 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:04.731498 28436 hybrid_clock.cc:648] HybridClock initialized: now 1790408644731457 us; error 28 us; skew 500 ppm
I20260926 07:44:04.731846 28436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:04 dist-test-slave-x492 krb5kdc[28423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:04.734901 28436 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:04.736388 28436 webserver.cc:460] Webserver started at http://127.11.9.126:37577/ using document root <none> and password file <none>
I20260926 07:44:04.737028 28436 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:04.737124 28436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:04.737450 28436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:04.740324 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "e029fb6c278d4a669c7b1f802391a55a"
format_stamp: "Formatted at 2026-09-26 07:44:04 on dist-test-slave-x492"
server_key: "e199af032d5d7d6a925ee8af3e7c4baf"
server_key_iv: "2708b0d5e530445d00bdabe9c1c9a987"
server_key_version: "encryptionkey@0"
I20260926 07:44:04.741016 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "e029fb6c278d4a669c7b1f802391a55a"
format_stamp: "Formatted at 2026-09-26 07:44:04 on dist-test-slave-x492"
server_key: "e199af032d5d7d6a925ee8af3e7c4baf"
server_key_iv: "2708b0d5e530445d00bdabe9c1c9a987"
server_key_version: "encryptionkey@0"
I20260926 07:44:04.746061 28436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 07:44:04.749362 28451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:04.750674 28436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:44:04.750876 28436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "e029fb6c278d4a669c7b1f802391a55a"
format_stamp: "Formatted at 2026-09-26 07:44:04 on dist-test-slave-x492"
server_key: "e199af032d5d7d6a925ee8af3e7c4baf"
server_key_iv: "2708b0d5e530445d00bdabe9c1c9a987"
server_key_version: "encryptionkey@0"
I20260926 07:44:04.751075 28436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:04.771163 28436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:04.775117 28436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:04.775415 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:04.794148 28436 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:32887
I20260926 07:44:04.794184 28502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:32887 every 8 connection(s)
I20260926 07:44:04.795564 28436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:04.799310 28503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:04.802610 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28436
I20260926 07:44:04.802855 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:04.803634 11301 external_mini_cluster.cc:1428] Setting key cbb3852907775740b874c28514566185
I20260926 07:44:04.812456 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: Bootstrap starting.
I20260926 07:44:04.816401 28503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:04.817826 28503 log.cc:824] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: Log is configured to *not* fsync() on all Append() calls
Sep 26 07:44:04 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:04.820967 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: No bootstrap required, opened a new log
I20260926 07:44:04.827718 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:04.805791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47888 (local address 127.11.9.126:32887)
0926 07:44:04.806596 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:04.806608 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:04.806650 (+ 42us) server_negotiation.cc:407] Connection header received
0926 07:44:04.808287 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:04.808311 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:04.808702 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:04.809041 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:04.810815 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:04.812093 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:04.813971 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:04.814652 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:04.814800 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:04.820051 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:04.820059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:04.820072 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:04.820123 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:04.823422 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:04.824293 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:04.824299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:04.824305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:04.824471 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:04.824892 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:04.824895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:04.824897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:04.825572 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:04.825798 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:04.826306 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 07:44:04.826712 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":396,"threads_started":1}
I20260926 07:44:04.830236 28503 raft_consensus.cc:348] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } }
I20260926 07:44:04.830760 28503 raft_consensus.cc:374] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:04.830873 28503 raft_consensus.cc:729] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e029fb6c278d4a669c7b1f802391a55a, State: Initialized, Role: FOLLOWER
I20260926 07:44:04.831480 28503 consensus_queue.cc:260] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } }
I20260926 07:44:04.831717 28503 raft_consensus.cc:388] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:04.831849 28503 raft_consensus.cc:482] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:04.832083 28503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:04.834324 28503 raft_consensus.cc:504] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } }
I20260926 07:44:04.834919 28503 leader_election.cc:302] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e029fb6c278d4a669c7b1f802391a55a; no voters:
I20260926 07:44:04.835549 28503 leader_election.cc:288] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:04.835690 28508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:04.837356 28503 sys_catalog.cc:564] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:04.838964 28508 raft_consensus.cc:686] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [term 1 LEADER]: Becoming Leader. State: Replica: e029fb6c278d4a669c7b1f802391a55a, State: Running, Role: LEADER
I20260926 07:44:04.839695 28508 consensus_queue.cc:237] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } }
I20260926 07:44:04.842926 28510 sys_catalog.cc:455] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e029fb6c278d4a669c7b1f802391a55a. Latest consensus state: current_term: 1 leader_uuid: "e029fb6c278d4a669c7b1f802391a55a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } } }
I20260926 07:44:04.843488 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e029fb6c278d4a669c7b1f802391a55a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e029fb6c278d4a669c7b1f802391a55a" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 32887 } } }
I20260926 07:44:04.843258 28510 sys_catalog.cc:458] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:04.843708 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:04.845683 28520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:04.850518 28520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:04.859905 28520 catalog_manager.cc:1347] Generated new cluster ID: 5aa52d7b7d89405aa8f8f05b124ed7fe
I20260926 07:44:04.860019 28520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:04.884830 28520 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:04.886281 28520 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:04.894084 28520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: Generated new TSK 0
I20260926 07:44:04.894953 28520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:44:04.968803 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:05.322371 28529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:05.322623 28529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:05.322781 28529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:05.330026 28529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:05.330149 28529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:05.330328 28529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:05.340003 28529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:05.341312 28529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:05.342567 28529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:05.355636 28535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:05.355665 28537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:05.355955 28534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:05.356786 28529 server_base.cc:1034] running on GCE node
I20260926 07:44:05.357421 28529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:05.358278 28529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:05.359508 28529 hybrid_clock.cc:648] HybridClock initialized: now 1790408645359472 us; error 57 us; skew 500 ppm
I20260926 07:44:05.359808 28529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:05 dist-test-slave-x492 krb5kdc[28423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:05.362958 28529 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:44:05.364315 28529 webserver.cc:460] Webserver started at http://127.11.9.65:34731/ using document root <none> and password file <none>
I20260926 07:44:05.364899 28529 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:05.365008 28529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:05.365403 28529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:05.368134 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "0989f64fbca14f5e8b7ecd08823e8388"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f8f868bf2301d555b365b960f16e8f40"
server_key_iv: "990f13538800b03cde128d23339d9542"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.368893 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "0989f64fbca14f5e8b7ecd08823e8388"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f8f868bf2301d555b365b960f16e8f40"
server_key_iv: "990f13538800b03cde128d23339d9542"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.373762 28529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 07:44:05.377137 28544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.378463 28529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:05.378625 28529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "0989f64fbca14f5e8b7ecd08823e8388"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f8f868bf2301d555b365b960f16e8f40"
server_key_iv: "990f13538800b03cde128d23339d9542"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.378811 28529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:05.394484 28529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:05.398680 28529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:05.398957 28529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:05.400251 28529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:05.401543 28529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:05.401613 28529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.401695 28529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:05.401739 28529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.433887 28529 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:43143
I20260926 07:44:05.433940 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:43143 every 8 connection(s)
I20260926 07:44:05.435176 28529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:05.436470 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28529
I20260926 07:44:05.436693 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:05.437532 11301 external_mini_cluster.cc:1428] Setting key d2d24295092bff7f994f934adb44a56a
Sep 26 07:44:05 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:05.456457 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:05.438840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:35817 (local address 127.11.9.126:32887)
0926 07:44:05.439351 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:05.439355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:05.440564 (+ 1209us) server_negotiation.cc:407] Connection header received
0926 07:44:05.442514 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:05.442520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:05.442601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:05.442765 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:05.444210 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.444950 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.446608 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.447019 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.447158 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:05.450531 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:05.450536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:05.450539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:05.450572 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:05.453237 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:05.453944 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:05.453950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:05.453953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:05.454079 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:05.454589 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:05.454594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:05.454598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:05.454920 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:05.455076 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:05.455853 (+ 777us) server_negotiation.cc:299] Negotiation successful
0926 07:44:05.456076 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":265,"threads_started":1}
I20260926 07:44:05.457636 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:05.439523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:32887 (local address 127.11.9.65:35817)
0926 07:44:05.440426 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:05.440470 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:05.442260 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:05.442994 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:05.443003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:05.443504 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:05.444047 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:05.444063 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.445105 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.445110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:05.446466 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:05.446478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.447256 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.447267 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:05.447501 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:05.448188 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:05.448209 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:05.450326 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:05.453367 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:05.453376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:05.453393 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:05.453817 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:05.454190 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:05.454194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:05.454198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:05.454473 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:05.455047 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:05.455055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:05.455455 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 07:44:05.455741 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 07:44:05.456306 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":390,"threads_started":1}
I20260926 07:44:05.459293 28657 heartbeater.cc:347] Connected to a master server at 127.11.9.126:32887
I20260926 07:44:05.459656 28657 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:05.460626 28657 heartbeater.cc:511] Master 127.11.9.126:32887 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:44:05.462997 28468 ts_manager.cc:194] Registered new tserver with Master: 0989f64fbca14f5e8b7ecd08823e8388 (127.11.9.65:43143)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
I20260926 07:44:05.464686 28468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:35817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:44:05.492926 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:05.816721 28663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:05.816977 28663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:05.817130 28663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:05.824517 28663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:05.824630 28663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:05.825011 28663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:05.834947 28663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:05.836206 28663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:05.837843 28663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:05.851379 28669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:05.851402 28671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:05.851473 28668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:05.852078 28663 server_base.cc:1034] running on GCE node
I20260926 07:44:05.852703 28663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:05.853637 28663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:05.854864 28663 hybrid_clock.cc:648] HybridClock initialized: now 1790408645854829 us; error 55 us; skew 500 ppm
I20260926 07:44:05.855273 28663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:05 dist-test-slave-x492 krb5kdc[28423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:05.858335 28663 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:44:05.859736 28663 webserver.cc:460] Webserver started at http://127.11.9.66:35635/ using document root <none> and password file <none>
I20260926 07:44:05.860387 28663 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:05.860527 28663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:05.860827 28663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:05.863703 28663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "be482fbd1777448686f07c39b4713429"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f622b19e16b630432a6856d1c045db47"
server_key_iv: "bfb41bd3589e80ba94141ea30b0f61ee"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.864460 28663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "be482fbd1777448686f07c39b4713429"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f622b19e16b630432a6856d1c045db47"
server_key_iv: "bfb41bd3589e80ba94141ea30b0f61ee"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.869441 28663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 07:44:05.873062 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.874408 28663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:05.874562 28663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "be482fbd1777448686f07c39b4713429"
format_stamp: "Formatted at 2026-09-26 07:44:05 on dist-test-slave-x492"
server_key: "f622b19e16b630432a6856d1c045db47"
server_key_iv: "bfb41bd3589e80ba94141ea30b0f61ee"
server_key_version: "encryptionkey@0"
I20260926 07:44:05.874758 28663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:05.912556 28663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:05.917057 28663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:05.917379 28663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:05.918347 28663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:05.919755 28663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:05.919863 28663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.920003 28663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:05.920073 28663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:05.953765 28663 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:46271
I20260926 07:44:05.953850 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:46271 every 8 connection(s)
I20260926 07:44:05.955052 28663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:05.965072 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28663
I20260926 07:44:05.965281 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:05.965762 11301 external_mini_cluster.cc:1428] Setting key dc089bb43c9c1a6900427cfbea6ff16d
Sep 26 07:44:05 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:05.977258 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:05.958546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:32887 (local address 127.11.9.66:48227)
0926 07:44:05.959267 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:05.959312 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:05.961124 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:05.961937 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:05.961945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:05.962456 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:05.963026 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:05.963048 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.964149 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.964154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:05.965649 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:05.965661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.966507 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.966513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:05.966745 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:05.967403 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:05.967425 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:05.969600 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:05.972585 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:05.972594 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:05.972611 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:05.973035 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:05.973450 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:05.973455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:05.973459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:05.973650 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:05.974176 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:05.974184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:05.974670 (+ 486us) client_negotiation.cc:769] Sending connection context
0926 07:44:05.974997 (+ 327us) client_negotiation.cc:240] Negotiation successful
0926 07:44:05.975630 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":285,"threads_started":1}
I20260926 07:44:05.978590 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:05.957976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:48227 (local address 127.11.9.126:32887)
0926 07:44:05.958436 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:05.958442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:05.959431 (+ 989us) server_negotiation.cc:407] Connection header received
0926 07:44:05.961575 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:05.961579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:05.961648 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:05.961733 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:05.963181 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.963986 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.965797 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:05.966282 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:05.966422 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:05.969835 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:05.969840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:05.969844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:05.969875 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:05.972441 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:05.973200 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:05.973206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:05.973209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:05.973339 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:05.973773 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:05.973778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:05.973781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:05.974041 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:05.977252 (+ 3211us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:05.977519 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 07:44:05.977902 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":210,"threads_started":1}
I20260926 07:44:05.980367 28791 heartbeater.cc:347] Connected to a master server at 127.11.9.126:32887
I20260926 07:44:05.980734 28791 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:05.981695 28791 heartbeater.cc:511] Master 127.11.9.126:32887 requested a full tablet report, sending...
I20260926 07:44:05.983568 28468 ts_manager.cc:194] Registered new tserver with Master: be482fbd1777448686f07c39b4713429 (127.11.9.66:46271)
I20260926 07:44:05.984702 28468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:48227
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:44:06.024366 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:06.349121 28797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:06.349443 28797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:06.349542 28797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:06.356827 28797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:06.356925 28797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:06.357035 28797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:06.366401 28797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:32887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:06.367491 28797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:06.368739 28797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:06.381378 28802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:06.381843 28803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:06.381848 28805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:06.382666 28797 server_base.cc:1034] running on GCE node
I20260926 07:44:06.383316 28797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:06.384227 28797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:06.385440 28797 hybrid_clock.cc:648] HybridClock initialized: now 1790408646385394 us; error 59 us; skew 500 ppm
I20260926 07:44:06.385747 28797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:06 dist-test-slave-x492 krb5kdc[28423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:06.388708 28797 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:44:06.390177 28797 webserver.cc:460] Webserver started at http://127.11.9.67:42591/ using document root <none> and password file <none>
I20260926 07:44:06.390857 28797 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:06.391006 28797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:06.391331 28797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:06.394153 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "f573a29d457447d19f3e2c55b13367bb"
format_stamp: "Formatted at 2026-09-26 07:44:06 on dist-test-slave-x492"
server_key: "0eb8419fdfdef3669532da0b022243f7"
server_key_iv: "a455fa3ea164a24fa0cd4bf7d29c4d8a"
server_key_version: "encryptionkey@0"
I20260926 07:44:06.394958 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "f573a29d457447d19f3e2c55b13367bb"
format_stamp: "Formatted at 2026-09-26 07:44:06 on dist-test-slave-x492"
server_key: "0eb8419fdfdef3669532da0b022243f7"
server_key_iv: "a455fa3ea164a24fa0cd4bf7d29c4d8a"
server_key_version: "encryptionkey@0"
I20260926 07:44:06.399863 28797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260926 07:44:06.403156 28812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:06.404374 28797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 07:44:06.404539 28797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "f573a29d457447d19f3e2c55b13367bb"
format_stamp: "Formatted at 2026-09-26 07:44:06 on dist-test-slave-x492"
server_key: "0eb8419fdfdef3669532da0b022243f7"
server_key_iv: "a455fa3ea164a24fa0cd4bf7d29c4d8a"
server_key_version: "encryptionkey@0"
I20260926 07:44:06.404726 28797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:06.423211 28797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:06.426956 28797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:06.427241 28797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:06.428189 28797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:06.429543 28797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:06.429644 28797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:06.429782 28797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:06.429852 28797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:06.463138 28797 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:40289
I20260926 07:44:06.463184 28924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:40289 every 8 connection(s)
I20260926 07:44:06.464401 28797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:06.467509 28657 heartbeater.cc:503] Master 127.11.9.126:32887 was elected leader, sending a full tablet report...
I20260926 07:44:06.467832 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28797
I20260926 07:44:06.468034 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:06.468611 11301 external_mini_cluster.cc:1428] Setting key 24926bb5f5f4d94cbf18f021280869dd
Sep 26 07:44:06 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:06.484704 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.466946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:33269 (local address 127.11.9.126:32887)
0926 07:44:06.467119 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.467123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.468571 (+ 1448us) server_negotiation.cc:407] Connection header received
0926 07:44:06.471006 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.471010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.471104 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.471209 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:06.472906 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.473693 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.475157 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.475531 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.475620 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.478793 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:06.478798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:06.478801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:06.478831 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:06.481551 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:06.482240 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:06.482245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:06.482248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:06.482403 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:06.482846 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:06.482850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:06.482852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:06.483217 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:06.483384 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.484152 (+ 768us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.484430 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:44:06.485622 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.467486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:32887 (local address 127.11.9.67:33269)
0926 07:44:06.468383 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.468443 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.470722 (+ 2279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.471522 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.471531 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.472049 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:06.472748 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.472764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.473820 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.473824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.475027 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.475038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.475702 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.475706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.475910 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.476499 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:06.476519 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:06.478621 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:06.481701 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:06.481709 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:06.481723 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:06.482117 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:06.482512 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:06.482517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:06.482521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:06.482702 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:06.483382 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:06.483391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:06.483760 (+ 369us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.484070 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.484545 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":354,"threads_started":1}
I20260926 07:44:06.486864 28925 heartbeater.cc:347] Connected to a master server at 127.11.9.126:32887
I20260926 07:44:06.487142 28925 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:06.487820 28925 heartbeater.cc:511] Master 127.11.9.126:32887 requested a full tablet report, sending...
I20260926 07:44:06.489137 28467 ts_manager.cc:194] Registered new tserver with Master: f573a29d457447d19f3e2c55b13367bb (127.11.9.67:40289)
I20260926 07:44:06.490010 28467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:33269
I20260926 07:44:06.496335 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:06.512760 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.500653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.11.9.126:32887)
0926 07:44:06.500800 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.500804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.501199 (+ 395us) server_negotiation.cc:407] Connection header received
0926 07:44:06.501351 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.501354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.501425 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.501503 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:06.502826 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.503492 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.504893 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.505432 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.505539 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.507873 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:06.507878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:06.507882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:06.507930 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:06.510525 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:06.511106 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:06.511109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:06.511111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:06.511275 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:06.511586 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:06.511589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:06.511591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:06.511909 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:06.512115 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.512353 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.512509 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 07:44:06.516516 28467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:06.523341 28467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:06.558509 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.535095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.11.9.65:43143)
0926 07:44:06.547501 (+ 12406us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.547514 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.548513 (+ 999us) server_negotiation.cc:407] Connection header received
0926 07:44:06.548949 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.548958 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.549270 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.549450 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:06.550758 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.552712 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.555569 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.556883 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.557019 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.557288 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.557990 (+ 702us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.558159 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12157,"thread_start_us":12406,"threads_started":1}
I20260926 07:44:06.558658 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.534899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43143 (local address 127.0.0.1:33942)
0926 07:44:06.548344 (+ 13445us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.548394 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.548669 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.549910 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.549927 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.549968 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:06.550509 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.550525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.552847 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.552853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.555403 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.555418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.557418 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.557424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.557706 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.557771 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.557936 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.558159 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13202,"mutex_wait_us":26,"thread_start_us":13060,"threads_started":1}
I20260926 07:44:06.559903 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.535533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:46271 (local address 127.0.0.1:34546)
0926 07:44:06.550024 (+ 14491us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.550054 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.550216 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.552102 (+ 1886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.552106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.552131 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:06.552790 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.552810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.554926 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.554931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.556819 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.556833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.559142 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.559150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.559326 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.559358 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.559453 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.559576 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14325,"thread_start_us":14798,"threads_started":1}
I20260926 07:44:06.560006 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.536081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.11.9.66:46271)
0926 07:44:06.551427 (+ 15346us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.551438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.551471 (+ 33us) server_negotiation.cc:407] Connection header received
0926 07:44:06.551567 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.551574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.551824 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.552008 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:06.553269 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.554762 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.557037 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.558605 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.558714 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.558943 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.559517 (+ 574us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.559690 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15180,"thread_start_us":15767,"threads_started":1}
I20260926 07:44:06.561640 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.551464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:40289 (local address 127.0.0.1:51544)
0926 07:44:06.552825 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.552846 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.552975 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.553809 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.553814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.553836 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:06.554268 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.554280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.556345 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.556350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.558604 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.558619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.560769 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.560778 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.560979 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.561009 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.561097 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.561250 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":3,"thread_start_us":1024,"threads_started":1}
I20260926 07:44:06.562328 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.549136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.11.9.67:40289)
0926 07:44:06.551826 (+ 2690us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.551834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.552999 (+ 1165us) server_negotiation.cc:407] Connection header received
0926 07:44:06.553102 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.553109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.553361 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.553535 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:06.554455 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.556184 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.558787 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.560548 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.560680 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.560886 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.561854 (+ 968us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.562003 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2143,"thread_start_us":1962,"threads_started":1}
I20260926 07:44:06.562860 28592 tablet_service.cc:1467] Processing CreateTablet for tablet b86046f136674bbc9475ed409bc1b7c4 (DEFAULT_TABLE table=test-table [id=ab54bb602fe145ecb81ee8899e789c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:06.563867 28726 tablet_service.cc:1467] Processing CreateTablet for tablet b86046f136674bbc9475ed409bc1b7c4 (DEFAULT_TABLE table=test-table [id=ab54bb602fe145ecb81ee8899e789c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:06.564366 28592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86046f136674bbc9475ed409bc1b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:06.564960 28726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86046f136674bbc9475ed409bc1b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:06.565915 28860 tablet_service.cc:1467] Processing CreateTablet for tablet b86046f136674bbc9475ed409bc1b7c4 (DEFAULT_TABLE table=test-table [id=ab54bb602fe145ecb81ee8899e789c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:06.567293 28860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86046f136674bbc9475ed409bc1b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:06.576727 28943 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Bootstrap starting.
I20260926 07:44:06.580451 28943 tablet_bootstrap.cc:654] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:06.581620 28943 log.cc:824] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:06.581873 28945 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Bootstrap starting.
I20260926 07:44:06.585763 28944 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Bootstrap starting.
I20260926 07:44:06.586026 28945 tablet_bootstrap.cc:654] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:06.587308 28945 log.cc:824] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:06.588023 28943 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: No bootstrap required, opened a new log
I20260926 07:44:06.588340 28943 ts_tablet_manager.cc:1397] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260926 07:44:06.589289 28944 tablet_bootstrap.cc:654] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:06.590471 28944 log.cc:824] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:06.591212 28945 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: No bootstrap required, opened a new log
I20260926 07:44:06.591483 28945 ts_tablet_manager.cc:1397] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260926 07:44:06.593348 28944 tablet_bootstrap.cc:492] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: No bootstrap required, opened a new log
I20260926 07:44:06.593603 28944 ts_tablet_manager.cc:1397] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 07:44:06.598747 28945 raft_consensus.cc:348] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.599340 28945 raft_consensus.cc:374] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:06.599086 28943 raft_consensus.cc:348] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.599570 28943 raft_consensus.cc:374] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:06.599560 28945 raft_consensus.cc:729] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f573a29d457447d19f3e2c55b13367bb, State: Initialized, Role: FOLLOWER
I20260926 07:44:06.599699 28943 raft_consensus.cc:729] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be482fbd1777448686f07c39b4713429, State: Initialized, Role: FOLLOWER
I20260926 07:44:06.600238 28945 consensus_queue.cc:260] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.600296 28943 consensus_queue.cc:260] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.601333 28925 heartbeater.cc:503] Master 127.11.9.126:32887 was elected leader, sending a full tablet report...
I20260926 07:44:06.602011 28945 ts_tablet_manager.cc:1428] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260926 07:44:06.605278 28791 heartbeater.cc:503] Master 127.11.9.126:32887 was elected leader, sending a full tablet report...
I20260926 07:44:06.605940 28943 ts_tablet_manager.cc:1428] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260926 07:44:06.606395 28944 raft_consensus.cc:348] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.606858 28944 raft_consensus.cc:374] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:06.606958 28944 raft_consensus.cc:729] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989f64fbca14f5e8b7ecd08823e8388, State: Initialized, Role: FOLLOWER
I20260926 07:44:06.607630 28944 consensus_queue.cc:260] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.609397 28944 ts_tablet_manager.cc:1428] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260926 07:44:06.687621 28658 tablet.cc:2369] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:06.707185 28792 tablet.cc:2369] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:06.716050 28926 tablet.cc:2369] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:06.903223 28950 raft_consensus.cc:482] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:06.903538 28950 raft_consensus.cc:504] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.905283 28950 leader_election.cc:288] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0989f64fbca14f5e8b7ecd08823e8388 (127.11.9.65:43143), f573a29d457447d19f3e2c55b13367bb (127.11.9.67:40289)
I20260926 07:44:06.912256 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.905629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38669 (local address 127.11.9.65:43143)
0926 07:44:06.905815 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.905820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.906382 (+ 562us) server_negotiation.cc:407] Connection header received
0926 07:44:06.906637 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.906643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.906726 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.906777 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:06.907587 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.908409 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.910488 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.911272 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.911402 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.911519 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.911874 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.912005 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 07:44:06.912348 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.905642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43143 (local address 127.11.9.66:38669)
0926 07:44:06.906234 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.906271 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.906441 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.906943 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.906946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.906998 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:06.907431 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.907447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.908533 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.908539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.910331 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.910343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.911509 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.911518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.911688 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.911712 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.911837 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.911963 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":24,"thread_start_us":329,"threads_started":1}
I20260926 07:44:06.912309 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.906467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:38617 (local address 127.11.9.67:40289)
0926 07:44:06.906643 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:06.906648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:06.907015 (+ 367us) server_negotiation.cc:407] Connection header received
0926 07:44:06.907148 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:06.907153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:06.907232 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:06.907338 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:06.907899 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.908701 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.910539 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.911298 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.911447 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.911558 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:06.911924 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 07:44:06.912056 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:44:06.912984 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:06.906309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:40289 (local address 127.11.9.66:38617)
0926 07:44:06.906912 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:06.906926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:06.907027 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:06.907347 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:06.907353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:06.907372 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:06.907757 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.907767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.908828 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.908831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.910363 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:06.910374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:06.911538 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:06.911546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:06.911716 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:06.911745 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 07:44:06.912517 (+ 772us) client_negotiation.cc:240] Negotiation successful
0926 07:44:06.912621 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":35,"thread_start_us":297,"threads_started":1}
I20260926 07:44:06.913583 28612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86046f136674bbc9475ed409bc1b7c4" candidate_uuid: "be482fbd1777448686f07c39b4713429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989f64fbca14f5e8b7ecd08823e8388" is_pre_election: true
I20260926 07:44:06.913869 28880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86046f136674bbc9475ed409bc1b7c4" candidate_uuid: "be482fbd1777448686f07c39b4713429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f573a29d457447d19f3e2c55b13367bb" is_pre_election: true
I20260926 07:44:06.914093 28612 raft_consensus.cc:2454] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be482fbd1777448686f07c39b4713429 in term 0.
I20260926 07:44:06.914351 28880 raft_consensus.cc:2454] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be482fbd1777448686f07c39b4713429 in term 0.
I20260926 07:44:06.914686 28680 leader_election.cc:302] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989f64fbca14f5e8b7ecd08823e8388, be482fbd1777448686f07c39b4713429; no voters:
I20260926 07:44:06.915009 28950 raft_consensus.cc:2781] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:06.915164 28950 raft_consensus.cc:482] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:06.915267 28950 raft_consensus.cc:3037] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:06.917467 28950 raft_consensus.cc:504] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.918009 28950 leader_election.cc:288] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [CANDIDATE]: Term 1 election: Requested vote from peers 0989f64fbca14f5e8b7ecd08823e8388 (127.11.9.65:43143), f573a29d457447d19f3e2c55b13367bb (127.11.9.67:40289)
I20260926 07:44:06.918404 28612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86046f136674bbc9475ed409bc1b7c4" candidate_uuid: "be482fbd1777448686f07c39b4713429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989f64fbca14f5e8b7ecd08823e8388"
I20260926 07:44:06.918529 28880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86046f136674bbc9475ed409bc1b7c4" candidate_uuid: "be482fbd1777448686f07c39b4713429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f573a29d457447d19f3e2c55b13367bb"
I20260926 07:44:06.918581 28612 raft_consensus.cc:3037] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:06.918735 28880 raft_consensus.cc:3037] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:06.920763 28612 raft_consensus.cc:2454] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be482fbd1777448686f07c39b4713429 in term 1.
I20260926 07:44:06.920877 28880 raft_consensus.cc:2454] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be482fbd1777448686f07c39b4713429 in term 1.
I20260926 07:44:06.921123 28680 leader_election.cc:302] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989f64fbca14f5e8b7ecd08823e8388, be482fbd1777448686f07c39b4713429; no voters:
I20260926 07:44:06.921403 28950 raft_consensus.cc:2781] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:06.921830 28950 raft_consensus.cc:686] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 1 LEADER]: Becoming Leader. State: Replica: be482fbd1777448686f07c39b4713429, State: Running, Role: LEADER
I20260926 07:44:06.922277 28950 consensus_queue.cc:237] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } }
I20260926 07:44:06.926898 28468 catalog_manager.cc:5495] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 reported cstate change: term changed from 0 to 1, leader changed from <none> to be482fbd1777448686f07c39b4713429 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "be482fbd1777448686f07c39b4713429" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be482fbd1777448686f07c39b4713429" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260926 07:44:07.012987 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:07.007802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.11.9.66:46271)
0926 07:44:07.008036 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:07.008040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:07.008076 (+ 36us) server_negotiation.cc:407] Connection header received
0926 07:44:07.008287 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:07.008291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:07.008367 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:07.008541 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:07.008848 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.009620 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.010989 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.011375 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.011465 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:07.011910 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:07.012394 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:07.012503 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:07.012611 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 07:44:07.012674 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 07:44:07.020901 28880 raft_consensus.cc:1261] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 1 FOLLOWER]: Refusing update from remote peer be482fbd1777448686f07c39b4713429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:07.020948 28612 raft_consensus.cc:1261] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 1 FOLLOWER]: Refusing update from remote peer be482fbd1777448686f07c39b4713429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:07.021867 28954 consensus_queue.cc:1035] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f573a29d457447d19f3e2c55b13367bb" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:07.022218 28950 consensus_queue.cc:1035] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989f64fbca14f5e8b7ecd08823e8388" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:07.040778 28957 mvcc.cc:205] Tried to move back new op lower bound from 7333513818188521472 to 7333513817797414912. Current Snapshot: MvccSnapshot[applied={T|T < 7333513818188521472 or (T in {7333513818188521472})}]
I20260926 07:44:07.043964 28958 mvcc.cc:205] Tried to move back new op lower bound from 7333513818188521472 to 7333513817797414912. Current Snapshot: MvccSnapshot[applied={T|T < 7333513818188521472}]
I20260926 07:44:07.057264 28467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:07.057590 28467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:07.062103 28467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e029fb6c278d4a669c7b1f802391a55a: Sending DeleteTablet for 3 replicas of tablet b86046f136674bbc9475ed409bc1b7c4
I20260926 07:44:07.063366 28592 tablet_service.cc:1514] Processing DeleteTablet for tablet b86046f136674bbc9475ed409bc1b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:07 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:33942
I20260926 07:44:07.063849 28860 tablet_service.cc:1514] Processing DeleteTablet for tablet b86046f136674bbc9475ed409bc1b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:07 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:51544
I20260926 07:44:07.064364 28967 tablet_replica.cc:330] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: stopping tablet replica
I20260926 07:44:07.064707 28968 tablet_replica.cc:330] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: stopping tablet replica
I20260926 07:44:07.064824 28967 raft_consensus.cc:2229] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:07.065100 28968 raft_consensus.cc:2229] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:07.065372 28967 raft_consensus.cc:2258] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:07.065418 28726 tablet_service.cc:1514] Processing DeleteTablet for tablet b86046f136674bbc9475ed409bc1b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:07 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:34546
I20260926 07:44:07.065577 28968 raft_consensus.cc:2258] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:07.067281 28967 ts_tablet_manager.cc:1905] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:07.068919 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28529
I20260926 07:44:07.075476 28967 ts_tablet_manager.cc:1918] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:07.075651 28967 log.cc:1196] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/b86046f136674bbc9475ed409bc1b7c4
I20260926 07:44:07.076261 28967 ts_tablet_manager.cc:1939] T b86046f136674bbc9475ed409bc1b7c4 P 0989f64fbca14f5e8b7ecd08823e8388: Deleting consensus metadata
I20260926 07:44:07.077297 28968 ts_tablet_manager.cc:1905] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:07.078442 28453 catalog_manager.cc:4841] TS 0989f64fbca14f5e8b7ecd08823e8388 (127.11.9.65:43143): tablet b86046f136674bbc9475ed409bc1b7c4 (table test-table [id=ab54bb602fe145ecb81ee8899e789c2d]) successfully deleted
I20260926 07:44:07.088547 28969 tablet_replica.cc:330] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: stopping tablet replica
I20260926 07:44:07.089036 28969 raft_consensus.cc:2229] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 1 LEADER]: Raft consensus shutting down.
Sep 26 07:44:07 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:44:07.093674 28969 raft_consensus.cc:2258] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:07.097136 28969 ts_tablet_manager.cc:1905] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:07.100051 28968 ts_tablet_manager.cc:1918] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:07.100242 28968 log.cc:1196] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/b86046f136674bbc9475ed409bc1b7c4
I20260926 07:44:07.100875 28968 ts_tablet_manager.cc:1939] T b86046f136674bbc9475ed409bc1b7c4 P f573a29d457447d19f3e2c55b13367bb: Deleting consensus metadata
I20260926 07:44:07.102872 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:07.071352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.11.9.65:43143)
0926 07:44:07.071542 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:07.071548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:07.080474 (+ 8926us) server_negotiation.cc:407] Connection header received
0926 07:44:07.080668 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:07.080673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:07.080772 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:07.080887 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:07.082653 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.083798 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.086934 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.088412 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.088575 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:07.095287 (+ 6712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:07.095305 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:07.095324 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:07.095377 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:07.099176 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.100058 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.100067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.100071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.100373 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.100832 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.100838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.100842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.101392 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:07.101671 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:07.102238 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 07:44:07.102489 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:44:07.102892 28453 catalog_manager.cc:4841] TS f573a29d457447d19f3e2c55b13367bb (127.11.9.67:40289): tablet b86046f136674bbc9475ed409bc1b7c4 (table test-table [id=ab54bb602fe145ecb81ee8899e789c2d]) successfully deleted
I20260926 07:44:07.104490 28652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:07.109802 28969 ts_tablet_manager.cc:1918] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:07.109984 28969 log.cc:1196] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/b86046f136674bbc9475ed409bc1b7c4
I20260926 07:44:07.110483 28969 ts_tablet_manager.cc:1939] T b86046f136674bbc9475ed409bc1b7c4 P be482fbd1777448686f07c39b4713429: Deleting consensus metadata
I20260926 07:44:07.112298 28454 catalog_manager.cc:4841] TS be482fbd1777448686f07c39b4713429 (127.11.9.66:46271): tablet b86046f136674bbc9475ed409bc1b7c4 (table test-table [id=ab54bb602fe145ecb81ee8899e789c2d]) successfully deleted
I20260926 07:44:07.198733 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28529
I20260926 07:44:07.222685 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28663
Sep 26 07:44:07 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:44:07.239516 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:07.223503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.11.9.66:46271)
0926 07:44:07.223691 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:07.223696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:07.223717 (+ 21us) server_negotiation.cc:407] Connection header received
0926 07:44:07.223879 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:07.223883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:07.223949 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:07.224064 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:07.225429 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.226227 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.227596 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.228052 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.228181 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:07.232997 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:07.233025 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:07.233042 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:07.233086 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:07.236439 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.237273 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.237292 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.237295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.237444 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.237895 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.237900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.237903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.238304 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:07.238656 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:07.238944 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 07:44:07.239190 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 07:44:07.240237 28786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:07.354031 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28663
I20260926 07:44:07.372226 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28797
Sep 26 07:44:07 dist-test-slave-x492 krb5kdc[28423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:44:07.387434 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:07.372972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.11.9.67:40289)
0926 07:44:07.373190 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:07.373197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:07.373218 (+ 21us) server_negotiation.cc:407] Connection header received
0926 07:44:07.373412 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:07.373417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:07.373493 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:07.373596 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:07.374951 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.375675 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.376978 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:07.377591 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:07.377730 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:07.381875 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:07.381889 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:07.381904 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:07.381939 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:07.384572 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.385395 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.385404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.385408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.385550 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:07.385919 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:07.385924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:07.385927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:07.386273 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:07.386556 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:07.386887 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 07:44:07.387097 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:44:07.388078 28920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:07.476029 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28797
I20260926 07:44:07.492630 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28436
I20260926 07:44:07.493949 28498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:07.604420 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28436
2026-09-26T07:44:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4572 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T07:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:07Z Disabled control of system clock
I20260926 07:44:07.678203 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:40869
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:08.015262 28978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:08.015554 28978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:08.015712 28978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:08.023075 28978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:08.023222 28978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:08.023311 28978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:08.023408 28978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:08.023484 28978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:08.032646 28978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:40869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:08.034076 28978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:08.035840 28978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:08.047438 28984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:08.047396 28986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:08.047473 28983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:08.048236 28978 server_base.cc:1034] running on GCE node
I20260926 07:44:08.049017 28978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:08.050482 28978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:08.051739 28978 hybrid_clock.cc:648] HybridClock initialized: now 1790408648051689 us; error 31 us; skew 500 ppm
I20260926 07:44:08.052213 28978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:08.060159 28978 webserver.cc:460] Webserver started at http://127.11.9.126:43587/ using document root <none> and password file <none>
I20260926 07:44:08.060921 28978 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:08.061044 28978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:08.061471 28978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:08.064667 28978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "7e2489ae0fe94742971cb5a271de6de3"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "c02af90fb2cdbb3c0feb842b4089f8d3"
server_key_iv: "29d3cd41259086b741101bcf7bfd2860"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.065608 28978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "7e2489ae0fe94742971cb5a271de6de3"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "c02af90fb2cdbb3c0feb842b4089f8d3"
server_key_iv: "29d3cd41259086b741101bcf7bfd2860"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.071225 28978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:08.075006 28992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:08.076375 28978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:44:08.076583 28978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "7e2489ae0fe94742971cb5a271de6de3"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "c02af90fb2cdbb3c0feb842b4089f8d3"
server_key_iv: "29d3cd41259086b741101bcf7bfd2860"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.076800 28978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:08.092598 28978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:08.093461 28978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:08.093744 28978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:08.111817 28978 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:40869
I20260926 07:44:08.111845 29043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:40869 every 8 connection(s)
I20260926 07:44:08.113209 28978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:08.116808 29044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:08.121133 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 28978
I20260926 07:44:08.121343 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:08.121913 11301 external_mini_cluster.cc:1428] Setting key ea00d32598e7911625c1ae016aa3d2f9
I20260926 07:44:08.128578 29044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: Bootstrap starting.
I20260926 07:44:08.132197 29044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:08.133474 29044 log.cc:824] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:08.136338 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:08.123918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.11.9.126:40869)
0926 07:44:08.124899 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:08.124910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:08.124955 (+ 45us) server_negotiation.cc:407] Connection header received
0926 07:44:08.127091 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:08.127115 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:08.127574 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:08.127906 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:08.128546 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.130016 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.131968 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.132715 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.132850 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:08.134168 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:08.134180 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:08.134193 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:08.134726 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:08.134800 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:08.134821 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:08.134937 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:08.135163 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 07:44:08.135456 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":372,"threads_started":1}
I20260926 07:44:08.136391 29044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: No bootstrap required, opened a new log
I20260926 07:44:08.144408 29044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } }
I20260926 07:44:08.144776 29044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:08.144896 29044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2489ae0fe94742971cb5a271de6de3, State: Initialized, Role: FOLLOWER
I20260926 07:44:08.145538 29044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } }
I20260926 07:44:08.145776 29044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:08.145906 29044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:08.146073 29044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:08.148627 29044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } }
I20260926 07:44:08.149113 29044 leader_election.cc:302] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e2489ae0fe94742971cb5a271de6de3; no voters:
I20260926 07:44:08.149742 29044 leader_election.cc:288] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:08.149905 29049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:08.150635 29049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [term 1 LEADER]: Becoming Leader. State: Replica: 7e2489ae0fe94742971cb5a271de6de3, State: Running, Role: LEADER
I20260926 07:44:08.151124 29049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } }
I20260926 07:44:08.151737 29044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:08.153954 29051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e2489ae0fe94742971cb5a271de6de3. Latest consensus state: current_term: 1 leader_uuid: "7e2489ae0fe94742971cb5a271de6de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } } }
I20260926 07:44:08.154426 29051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:08.154263 29050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e2489ae0fe94742971cb5a271de6de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2489ae0fe94742971cb5a271de6de3" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 40869 } } }
I20260926 07:44:08.154913 29050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:08.157092 29058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:08.162072 29058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:08.171223 29058 catalog_manager.cc:1347] Generated new cluster ID: d79d756f50864f80bda8554d75e69c15
I20260926 07:44:08.171345 29058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:08.209098 29058 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:08.210983 29058 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:08.224833 29058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: Generated new TSK 0
I20260926 07:44:08.225821 29058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:08.243227 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--builtin_ntp_servers=127.11.9.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:08.581979 29068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:08.582257 29068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:08.582407 29068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:08.589567 29068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:08.589663 29068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:08.589843 29068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:08.598724 29068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:08.599946 29068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:08.601698 29068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:08.614070 29076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:08.614202 29073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:08.614204 29074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:08.615234 29068 server_base.cc:1034] running on GCE node
I20260926 07:44:08.615813 29068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:08.616691 29068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:08.617880 29068 hybrid_clock.cc:648] HybridClock initialized: now 1790408648617844 us; error 29 us; skew 500 ppm
I20260926 07:44:08.618181 29068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:08.620175 29068 webserver.cc:460] Webserver started at http://127.11.9.65:35469/ using document root <none> and password file <none>
I20260926 07:44:08.620811 29068 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:08.620894 29068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:08.621145 29068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:08.623849 29068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "6de8b16712554b11a8e5ed4c53fd2bd9"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "f058561eaa838611b67e97b499acd460"
server_key_iv: "01251ab9d85b6bca1eddd4bd0e0d3766"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.624553 29068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "6de8b16712554b11a8e5ed4c53fd2bd9"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "f058561eaa838611b67e97b499acd460"
server_key_iv: "01251ab9d85b6bca1eddd4bd0e0d3766"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.629336 29068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260926 07:44:08.632395 29082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:08.633668 29068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 07:44:08.633826 29068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "6de8b16712554b11a8e5ed4c53fd2bd9"
format_stamp: "Formatted at 2026-09-26 07:44:08 on dist-test-slave-x492"
server_key: "f058561eaa838611b67e97b499acd460"
server_key_iv: "01251ab9d85b6bca1eddd4bd0e0d3766"
server_key_version: "encryptionkey@0"
I20260926 07:44:08.634021 29068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:08.658291 29068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:08.659116 29068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:08.659390 29068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:08.660332 29068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:08.661765 29068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:08.661885 29068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:08.662000 29068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:08.662086 29068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:08.694921 29068 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:41701
I20260926 07:44:08.695001 29194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:41701 every 8 connection(s)
I20260926 07:44:08.696267 29068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:08.699494 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29068
I20260926 07:44:08.699680 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:08.700266 11301 external_mini_cluster.cc:1428] Setting key da727c3480a9ac3b9c54bd9eb386fe4a
I20260926 07:44:08.703392 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--builtin_ntp_servers=127.11.9.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:08.710598 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:08.698800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:41561 (local address 127.11.9.126:40869)
0926 07:44:08.699293 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:08.699298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:08.700675 (+ 1377us) server_negotiation.cc:407] Connection header received
0926 07:44:08.702891 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:08.702898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:08.702965 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:08.703056 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:08.704513 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.705351 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.707748 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.708144 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.708238 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:08.709879 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:08.709884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:08.709887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:08.709905 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:08.709946 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:08.709957 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:08.710021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:08.710280 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 07:44:08.710356 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":206,"threads_started":1}
I20260926 07:44:08.711988 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:08.699402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40869 (local address 127.11.9.65:41561)
0926 07:44:08.700493 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:08.700538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:08.702593 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:08.703327 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:08.703340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:08.703467 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:08.704311 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:08.704340 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.705498 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.705507 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:08.707602 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:08.707629 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:08.708372 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:08.708382 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:08.708727 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:08.709515 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:08.709521 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:08.709572 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:08.709686 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:08.710051 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:08.710065 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:08.710076 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 07:44:08.710259 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 07:44:08.710682 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":324,"threads_started":1}
I20260926 07:44:08.713635 29195 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40869
I20260926 07:44:08.713927 29195 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:08.714638 29195 heartbeater.cc:511] Master 127.11.9.126:40869 requested a full tablet report, sending...
I20260926 07:44:08.716542 29009 ts_manager.cc:194] Registered new tserver with Master: 6de8b16712554b11a8e5ed4c53fd2bd9 (127.11.9.65:41701)
I20260926 07:44:08.718031 29009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.65:41561
W20260926 07:44:09.027334 29199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:09.027609 29199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:09.027695 29199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:09.034813 29199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:09.034907 29199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:09.035012 29199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:09.044062 29199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:09.045388 29199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:09.047047 29199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:09.059967 29204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:09.060021 29205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:09.060047 29207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:09.060853 29199 server_base.cc:1034] running on GCE node
I20260926 07:44:09.061486 29199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:09.062355 29199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:09.063588 29199 hybrid_clock.cc:648] HybridClock initialized: now 1790408649063538 us; error 58 us; skew 500 ppm
I20260926 07:44:09.063903 29199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:09.066144 29199 webserver.cc:460] Webserver started at http://127.11.9.66:42631/ using document root <none> and password file <none>
I20260926 07:44:09.066804 29199 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:09.066927 29199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:09.067238 29199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:09.070086 29199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "cd072dd8b3374921b5c492f4520f14d9"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "ab78c91f0a39484b2396e5cfb83d7b91"
server_key_iv: "3af8072886fba15f691c188857560b80"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.070844 29199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "cd072dd8b3374921b5c492f4520f14d9"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "ab78c91f0a39484b2396e5cfb83d7b91"
server_key_iv: "3af8072886fba15f691c188857560b80"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.075999 29199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:09.079365 29213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.080657 29199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:44:09.080824 29199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "cd072dd8b3374921b5c492f4520f14d9"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "ab78c91f0a39484b2396e5cfb83d7b91"
server_key_iv: "3af8072886fba15f691c188857560b80"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.081022 29199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:09.102679 29199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:09.103463 29199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:09.103740 29199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:09.104748 29199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:09.106240 29199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:09.106356 29199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.106477 29199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:09.106559 29199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.140416 29199 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:42217
I20260926 07:44:09.140499 29325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:42217 every 8 connection(s)
I20260926 07:44:09.141866 29199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:09.148759 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29199
I20260926 07:44:09.148937 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:09.149469 11301 external_mini_cluster.cc:1428] Setting key 8152e3352013626109bccfe5921751bb
I20260926 07:44:09.152263 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--builtin_ntp_servers=127.11.9.84:40049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:09.155251 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.144538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:48683 (local address 127.11.9.126:40869)
0926 07:44:09.144765 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.144771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.146274 (+ 1503us) server_negotiation.cc:407] Connection header received
0926 07:44:09.148060 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.148065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.148133 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.148229 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:09.149415 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.150357 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.152080 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.152871 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.153018 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.154351 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:09.154360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:09.154368 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:09.154393 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:09.154438 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:09.154456 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:09.154555 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.154751 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.154859 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 07:44:09.156198 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.145137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40869 (local address 127.11.9.66:48683)
0926 07:44:09.146091 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.146125 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.147816 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.148428 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.148437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.148549 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:09.149236 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.149264 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.150518 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.150524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.151938 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.151951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.153071 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.153076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.153339 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.154039 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:09.154042 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:09.154086 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:09.154165 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:09.154551 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:09.154558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:09.154564 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.154695 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.154994 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":267,"threads_started":1}
I20260926 07:44:09.157613 29326 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40869
I20260926 07:44:09.157948 29326 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:09.158816 29326 heartbeater.cc:511] Master 127.11.9.126:40869 requested a full tablet report, sending...
I20260926 07:44:09.160287 29009 ts_manager.cc:194] Registered new tserver with Master: cd072dd8b3374921b5c492f4520f14d9 (127.11.9.66:42217)
I20260926 07:44:09.161087 29009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.66:48683
W20260926 07:44:09.472143 29329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:09.472443 29329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:09.472589 29329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:09.479801 29329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:09.479933 29329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:09.480106 29329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:09.489243 29329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:40049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:40869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:09.490476 29329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:09.492208 29329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:09.505432 29334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:09.505460 29335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:09.505769 29337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:09.506350 29329 server_base.cc:1034] running on GCE node
I20260926 07:44:09.506916 29329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:09.507807 29329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:09.509029 29329 hybrid_clock.cc:648] HybridClock initialized: now 1790408649508983 us; error 52 us; skew 500 ppm
I20260926 07:44:09.509398 29329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:09.511518 29329 webserver.cc:460] Webserver started at http://127.11.9.67:33787/ using document root <none> and password file <none>
I20260926 07:44:09.512176 29329 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:09.512305 29329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:09.512610 29329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:09.515465 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "841ea011b8fa1fd9d7773dee094b97bc"
server_key_iv: "aa1861847d2511769603b8d83eb87a69"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.516237 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "841ea011b8fa1fd9d7773dee094b97bc"
server_key_iv: "aa1861847d2511769603b8d83eb87a69"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.521081 29329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260926 07:44:09.524454 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.525727 29329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:09.525890 29329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b"
format_stamp: "Formatted at 2026-09-26 07:44:09 on dist-test-slave-x492"
server_key: "841ea011b8fa1fd9d7773dee094b97bc"
server_key_iv: "aa1861847d2511769603b8d83eb87a69"
server_key_version: "encryptionkey@0"
I20260926 07:44:09.526095 29329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:09.560896 29329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:09.561794 29329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:09.562084 29329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:09.563047 29329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:09.564587 29329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:09.564706 29329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.564828 29329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:09.564919 29329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:09.597528 29329 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:32777
I20260926 07:44:09.597581 29455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:32777 every 8 connection(s)
I20260926 07:44:09.598975 29329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:09.605870 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29329
I20260926 07:44:09.606068 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:09.606588 11301 external_mini_cluster.cc:1428] Setting key ae348a3b92d035f3fd5d17c42361bd96
I20260926 07:44:09.611375 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.601434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:43823 (local address 127.11.9.126:40869)
0926 07:44:09.601610 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.601615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.602991 (+ 1376us) server_negotiation.cc:407] Connection header received
0926 07:44:09.604791 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.604797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.604882 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.604986 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:09.606374 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.607187 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.608842 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.609323 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.609452 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.610728 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:09.610733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:09.610736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:09.610753 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:09.610812 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:09.610825 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:09.610895 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.611079 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.611149 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:44:09.612321 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.602002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:40869 (local address 127.11.9.67:43823)
0926 07:44:09.602851 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.602886 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.604533 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.605283 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.605291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.605371 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:09.606171 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.606196 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.607325 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.607329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.608702 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.608714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.609529 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.609537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.609762 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.610380 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:09.610383 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:09.610432 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:09.610552 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:09.610909 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:09.610916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:09.610921 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.611055 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.611337 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":256,"threads_started":1}
I20260926 07:44:09.613523 29456 heartbeater.cc:347] Connected to a master server at 127.11.9.126:40869
I20260926 07:44:09.613786 29456 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:09.614431 29456 heartbeater.cc:511] Master 127.11.9.126:40869 requested a full tablet report, sending...
I20260926 07:44:09.615695 29009 ts_manager.cc:194] Registered new tserver with Master: 721ebbf4bcbb423aa06b5eb2b9b85e9b (127.11.9.67:32777)
I20260926 07:44:09.616518 29009 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.67:43823
I20260926 07:44:09.621245 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:09.630702 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.625261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.11.9.126:40869)
0926 07:44:09.625405 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.625409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.625842 (+ 433us) server_negotiation.cc:407] Connection header received
0926 07:44:09.625969 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.625972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.626038 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.626123 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:09.626546 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.627210 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.628504 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.628916 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.629000 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.630111 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:09.630115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:09.630118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:09.630133 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:09.630179 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:09.630191 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:09.630297 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.630435 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.630499 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 07:44:09.634321 29009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:09.641044 29009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:09.662791 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.652650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:32777 (local address 127.0.0.1:34156)
0926 07:44:09.656086 (+ 3436us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.656111 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.656234 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.656765 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.656771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.656800 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:09.657193 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.657209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.658525 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.658533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.660439 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.660455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.661792 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.661798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.662058 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.662139 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.662290 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.662465 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":1417,"threads_started":1}
I20260926 07:44:09.663743 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.652873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.11.9.67:32777)
0926 07:44:09.656000 (+ 3127us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.656011 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.656196 (+ 185us) server_negotiation.cc:407] Connection header received
0926 07:44:09.656366 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.656374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.656575 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.656759 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:09.657379 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.658399 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.660605 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.661600 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.661742 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.662034 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.662335 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.662489 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2959,"thread_start_us":2812,"threads_started":1}
I20260926 07:44:09.667644 29391 tablet_service.cc:1467] Processing CreateTablet for tablet 3249a53b0d88495783bbde4934f01252 (DEFAULT_TABLE table=test-table [id=8536662fcf6240578c5fc9cb1064228d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:09.669220 29391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3249a53b0d88495783bbde4934f01252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:09.672101 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.655302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:42217 (local address 127.0.0.1:38660)
0926 07:44:09.663383 (+ 8081us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.663400 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.663501 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.666081 (+ 2580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.666089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.666125 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:09.666305 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.666314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.668001 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.668006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.669658 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.669677 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.671369 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.671375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.671563 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.671599 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.671689 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.671817 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7950,"thread_start_us":10726,"threads_started":1}
I20260926 07:44:09.672499 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.655877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.11.9.66:42217)
0926 07:44:09.665015 (+ 9138us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.665027 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.665065 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:44:09.665200 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.665212 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.665526 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.665770 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:09.666475 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.667832 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.669823 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.671209 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.671339 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.671564 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.671759 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.671926 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8973,"thread_start_us":8877,"threads_started":1}
I20260926 07:44:09.672906 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.652838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50332)
0926 07:44:09.655128 (+ 2290us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.655172 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.655368 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.667109 (+ 11741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.667115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.667140 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:09.667452 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.667463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.669263 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.669269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.670687 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.670699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.672293 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.672299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.672445 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.672467 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.672517 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.672663 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2132,"thread_start_us":2007,"threads_started":1}
I20260926 07:44:09.674117 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.653439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.11.9.65:41701)
0926 07:44:09.666513 (+ 13074us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.666521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.666549 (+ 28us) server_negotiation.cc:407] Connection header received
0926 07:44:09.666657 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.666664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.666903 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.667101 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:09.667635 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.669096 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.670864 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.672068 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.673292 (+ 1224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.673529 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.673610 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.673770 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12913,"thread_start_us":12818,"threads_started":1}
I20260926 07:44:09.675905 29261 tablet_service.cc:1467] Processing CreateTablet for tablet 3249a53b0d88495783bbde4934f01252 (DEFAULT_TABLE table=test-table [id=8536662fcf6240578c5fc9cb1064228d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:09.677701 29261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3249a53b0d88495783bbde4934f01252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:09.677520 29130 tablet_service.cc:1467] Processing CreateTablet for tablet 3249a53b0d88495783bbde4934f01252 (DEFAULT_TABLE table=test-table [id=8536662fcf6240578c5fc9cb1064228d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:09.678777 29130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3249a53b0d88495783bbde4934f01252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:09.683015 29474 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Bootstrap starting.
I20260926 07:44:09.687148 29474 tablet_bootstrap.cc:654] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:09.688357 29474 log.cc:824] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:09.690670 29476 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Bootstrap starting.
I20260926 07:44:09.691390 29474 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: No bootstrap required, opened a new log
I20260926 07:44:09.691648 29474 ts_tablet_manager.cc:1397] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:44:09.695101 29476 tablet_bootstrap.cc:654] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:09.696525 29476 log.cc:824] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:09.698042 29477 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Bootstrap starting.
I20260926 07:44:09.701558 29474 raft_consensus.cc:348] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.701983 29474 raft_consensus.cc:374] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:09.702082 29474 raft_consensus.cc:729] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721ebbf4bcbb423aa06b5eb2b9b85e9b, State: Initialized, Role: FOLLOWER
I20260926 07:44:09.702793 29474 consensus_queue.cc:260] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.705215 29477 tablet_bootstrap.cc:654] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:09.705950 29456 heartbeater.cc:503] Master 127.11.9.126:40869 was elected leader, sending a full tablet report...
I20260926 07:44:09.706442 29474 ts_tablet_manager.cc:1428] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260926 07:44:09.706418 29477 log.cc:824] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:09.708835 29477 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: No bootstrap required, opened a new log
I20260926 07:44:09.708959 29476 tablet_bootstrap.cc:492] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: No bootstrap required, opened a new log
I20260926 07:44:09.709127 29477 ts_tablet_manager.cc:1397] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.006s
I20260926 07:44:09.709257 29476 ts_tablet_manager.cc:1397] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20260926 07:44:09.717613 29476 raft_consensus.cc:348] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.717943 29476 raft_consensus.cc:374] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:09.718055 29476 raft_consensus.cc:729] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd072dd8b3374921b5c492f4520f14d9, State: Initialized, Role: FOLLOWER
I20260926 07:44:09.717826 29477 raft_consensus.cc:348] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.718299 29477 raft_consensus.cc:374] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:09.718406 29477 raft_consensus.cc:729] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de8b16712554b11a8e5ed4c53fd2bd9, State: Initialized, Role: FOLLOWER
I20260926 07:44:09.718665 29476 consensus_queue.cc:260] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.718986 29477 consensus_queue.cc:260] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.719650 29326 heartbeater.cc:503] Master 127.11.9.126:40869 was elected leader, sending a full tablet report...
I20260926 07:44:09.720001 29195 heartbeater.cc:503] Master 127.11.9.126:40869 was elected leader, sending a full tablet report...
I20260926 07:44:09.720485 29477 ts_tablet_manager.cc:1428] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 07:44:09.721532 29476 ts_tablet_manager.cc:1428] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
W20260926 07:44:09.850687 29457 tablet.cc:2369] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:09.893949 29327 tablet.cc:2369] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:09.896190 29481 raft_consensus.cc:482] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:09.896445 29481 raft_consensus.cc:504] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.898144 29481 leader_election.cc:288] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 721ebbf4bcbb423aa06b5eb2b9b85e9b (127.11.9.67:32777), 6de8b16712554b11a8e5ed4c53fd2bd9 (127.11.9.65:41701)
I20260926 07:44:09.904464 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.899111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:46367 (local address 127.11.9.67:32777)
0926 07:44:09.899316 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.899325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.899447 (+ 122us) server_negotiation.cc:407] Connection header received
0926 07:44:09.899674 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.899678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.899738 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.899855 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:09.900328 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.901044 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.902842 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.903601 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.903749 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.903858 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.904104 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.904236 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:44:09.904637 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.898541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:32777 (local address 127.11.9.66:46367)
0926 07:44:09.899330 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.899352 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.899527 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.899872 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.899877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.899902 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:09.900192 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.900201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.901194 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.901201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.902652 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.902664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.903785 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.903792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.903954 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.903977 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.904079 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.904205 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":427,"threads_started":1}
I20260926 07:44:09.904786 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.898681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:42353 (local address 127.11.9.65:41701)
0926 07:44:09.898926 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.898932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.899754 (+ 822us) server_negotiation.cc:407] Connection header received
0926 07:44:09.900167 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.900171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.900237 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.900341 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:09.900851 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.901578 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.903204 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.903936 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.904086 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.904171 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.904415 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.904534 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 07:44:09.904866 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.898602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.11.9.66:42353)
0926 07:44:09.899665 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:09.899680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:09.899783 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:09.900352 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:09.900357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:09.900378 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:09.900697 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.900705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.901703 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.901708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.903039 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:09.903050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.904109 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.904115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:09.904271 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.904293 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 07:44:09.904391 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 07:44:09.904509 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":724,"threads_started":1}
I20260926 07:44:09.905691 29411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3249a53b0d88495783bbde4934f01252" candidate_uuid: "cd072dd8b3374921b5c492f4520f14d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" is_pre_election: true
I20260926 07:44:09.905750 29150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3249a53b0d88495783bbde4934f01252" candidate_uuid: "cd072dd8b3374921b5c492f4520f14d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" is_pre_election: true
I20260926 07:44:09.906180 29411 raft_consensus.cc:2454] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd072dd8b3374921b5c492f4520f14d9 in term 0.
I20260926 07:44:09.906191 29150 raft_consensus.cc:2454] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd072dd8b3374921b5c492f4520f14d9 in term 0.
I20260926 07:44:09.906783 29216 leader_election.cc:302] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de8b16712554b11a8e5ed4c53fd2bd9, cd072dd8b3374921b5c492f4520f14d9; no voters:
I20260926 07:44:09.907155 29481 raft_consensus.cc:2781] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:09.907300 29481 raft_consensus.cc:482] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:09.907405 29481 raft_consensus.cc:3037] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:09.909508 29481 raft_consensus.cc:504] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.910032 29481 leader_election.cc:288] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [CANDIDATE]: Term 1 election: Requested vote from peers 721ebbf4bcbb423aa06b5eb2b9b85e9b (127.11.9.67:32777), 6de8b16712554b11a8e5ed4c53fd2bd9 (127.11.9.65:41701)
I20260926 07:44:09.910410 29411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3249a53b0d88495783bbde4934f01252" candidate_uuid: "cd072dd8b3374921b5c492f4520f14d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b"
I20260926 07:44:09.910437 29150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3249a53b0d88495783bbde4934f01252" candidate_uuid: "cd072dd8b3374921b5c492f4520f14d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9"
I20260926 07:44:09.910609 29411 raft_consensus.cc:3037] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:09.910617 29150 raft_consensus.cc:3037] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:09.913810 29411 raft_consensus.cc:2454] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd072dd8b3374921b5c492f4520f14d9 in term 1.
I20260926 07:44:09.913803 29150 raft_consensus.cc:2454] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd072dd8b3374921b5c492f4520f14d9 in term 1.
I20260926 07:44:09.914207 29215 leader_election.cc:302] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721ebbf4bcbb423aa06b5eb2b9b85e9b, cd072dd8b3374921b5c492f4520f14d9; no voters:
I20260926 07:44:09.914476 29481 raft_consensus.cc:2781] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:09.914904 29481 raft_consensus.cc:686] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 1 LEADER]: Becoming Leader. State: Replica: cd072dd8b3374921b5c492f4520f14d9, State: Running, Role: LEADER
I20260926 07:44:09.915375 29481 consensus_queue.cc:237] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } }
I20260926 07:44:09.919973 29009 catalog_manager.cc:5495] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd072dd8b3374921b5c492f4520f14d9 (127.11.9.66). New cstate: current_term: 1 leader_uuid: "cd072dd8b3374921b5c492f4520f14d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd072dd8b3374921b5c492f4520f14d9" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 42217 } health_report { overall_health: HEALTHY } } }
W20260926 07:44:09.948763 29196 tablet.cc:2369] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:09.958029 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:09.952444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.11.9.66:42217)
0926 07:44:09.952649 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:09.952655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:09.952756 (+ 101us) server_negotiation.cc:407] Connection header received
0926 07:44:09.952987 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:09.952992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:09.953101 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:09.953291 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:09.953635 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.954352 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.955881 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:09.956298 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:09.956425 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:09.956908 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:09.957457 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:09.957603 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:09.957702 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 07:44:09.957780 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 07:44:09.965682 29150 raft_consensus.cc:1261] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 1 FOLLOWER]: Refusing update from remote peer cd072dd8b3374921b5c492f4520f14d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:09.965699 29411 raft_consensus.cc:1261] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 1 FOLLOWER]: Refusing update from remote peer cd072dd8b3374921b5c492f4520f14d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:09.966667 29485 consensus_queue.cc:1035] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de8b16712554b11a8e5ed4c53fd2bd9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 41701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:09.966966 29481 consensus_queue.cc:1035] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "721ebbf4bcbb423aa06b5eb2b9b85e9b" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:09.979887 29488 mvcc.cc:205] Tried to move back new op lower bound from 7333513830250135552 to 7333513830056775680. Current Snapshot: MvccSnapshot[applied={T|T < 7333513830250135552}]
I20260926 07:44:09.996138 29009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:09.996382 29009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:09.999853 29009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e2489ae0fe94742971cb5a271de6de3: Sending DeleteTablet for 3 replicas of tablet 3249a53b0d88495783bbde4934f01252
I20260926 07:44:10.000855 29391 tablet_service.cc:1514] Processing DeleteTablet for tablet 3249a53b0d88495783bbde4934f01252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:09 UTC) from {username='slave'} at 127.0.0.1:34156
I20260926 07:44:10.000972 29130 tablet_service.cc:1514] Processing DeleteTablet for tablet 3249a53b0d88495783bbde4934f01252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:09 UTC) from {username='slave'} at 127.0.0.1:50332
I20260926 07:44:10.001186 29261 tablet_service.cc:1514] Processing DeleteTablet for tablet 3249a53b0d88495783bbde4934f01252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:09 UTC) from {username='slave'} at 127.0.0.1:38660
I20260926 07:44:10.002722 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29068
I20260926 07:44:10.022014 29499 tablet_replica.cc:330] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: stopping tablet replica
I20260926 07:44:10.022447 29499 raft_consensus.cc:2229] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:10.022989 29499 raft_consensus.cc:2258] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:10.024999 29498 tablet_replica.cc:330] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: stopping tablet replica
I20260926 07:44:10.025484 29498 raft_consensus.cc:2229] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:10.026001 29498 raft_consensus.cc:2258] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:10.027112 29499 ts_tablet_manager.cc:1905] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:10.028434 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.004435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.11.9.65:41701)
0926 07:44:10.004668 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:10.004677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:10.015020 (+ 10343us) server_negotiation.cc:407] Connection header received
0926 07:44:10.015606 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:10.015615 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:10.015755 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:10.015866 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:10.016724 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.018374 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.023414 (+ 5040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.024011 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.024088 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:10.026702 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:10.026711 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:10.026728 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:10.027475 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:10.027540 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:10.027572 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:10.027656 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:10.027951 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 07:44:10.028069 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:44:10.029722 29190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:10.031335 29500 tablet_replica.cc:330] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: stopping tablet replica
I20260926 07:44:10.031728 29500 raft_consensus.cc:2229] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:44:10.032467 29500 raft_consensus.cc:2258] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:10.034766 29498 ts_tablet_manager.cc:1905] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:10.035207 29500 ts_tablet_manager.cc:1905] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:10.042117 29498 ts_tablet_manager.cc:1918] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:10.042275 29498 log.cc:1196] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/3249a53b0d88495783bbde4934f01252
I20260926 07:44:10.042800 29498 ts_tablet_manager.cc:1939] T 3249a53b0d88495783bbde4934f01252 P 721ebbf4bcbb423aa06b5eb2b9b85e9b: Deleting consensus metadata
I20260926 07:44:10.044665 28994 catalog_manager.cc:4841] TS 721ebbf4bcbb423aa06b5eb2b9b85e9b (127.11.9.67:32777): tablet 3249a53b0d88495783bbde4934f01252 (table test-table [id=8536662fcf6240578c5fc9cb1064228d]) successfully deleted
I20260926 07:44:10.045454 29500 ts_tablet_manager.cc:1918] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:10.045763 29500 log.cc:1196] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/3249a53b0d88495783bbde4934f01252
I20260926 07:44:10.046883 29500 ts_tablet_manager.cc:1939] T 3249a53b0d88495783bbde4934f01252 P cd072dd8b3374921b5c492f4520f14d9: Deleting consensus metadata
I20260926 07:44:10.049348 28995 catalog_manager.cc:4841] TS cd072dd8b3374921b5c492f4520f14d9 (127.11.9.66:42217): tablet 3249a53b0d88495783bbde4934f01252 (table test-table [id=8536662fcf6240578c5fc9cb1064228d]) successfully deleted
I20260926 07:44:10.172909 29499 ts_tablet_manager.cc:1918] T 3249a53b0d88495783bbde4934f01252 P 6de8b16712554b11a8e5ed4c53fd2bd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:10.173138 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29068
I20260926 07:44:10.190532 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29199
W20260926 07:44:10.190550 28995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:41701 (error 108)
I20260926 07:44:10.192914 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.192088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50352)
0926 07:44:10.192597 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.192727 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":273,"threads_started":1}
W20260926 07:44:10.193253 28995 catalog_manager.cc:4584] TS 6de8b16712554b11a8e5ed4c53fd2bd9 (127.11.9.65:41701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3249a53b0d88495783bbde4934f01252: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
I20260926 07:44:10.198555 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.191658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.11.9.66:42217)
0926 07:44:10.191857 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:10.191861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:10.191878 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:44:10.191936 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:10.191940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:10.192013 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:10.192136 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:10.192611 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.193863 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.195108 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.195783 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.195918 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:10.196899 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:10.196913 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:10.196935 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:10.197782 (+ 847us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:10.197846 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:10.197872 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:10.197975 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:10.198087 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 07:44:10.198181 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:44:10.199345 29321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:10.246908 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.246441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50362)
0926 07:44:10.246680 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.246768 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 07:44:10.247951 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.247581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50372)
0926 07:44:10.247746 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.247807 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 07:44:10.308476 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.308006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50374)
0926 07:44:10.308234 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.308323 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 07:44:10.309623 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.309257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50382)
0926 07:44:10.309409 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.309494 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 07:44:10.313215 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29199
I20260926 07:44:10.332628 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29329
I20260926 07:44:10.338850 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.333381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.11.9.67:32777)
0926 07:44:10.333545 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:10.333550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:10.333654 (+ 104us) server_negotiation.cc:407] Connection header received
0926 07:44:10.333804 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:10.333807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:10.333886 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:10.333972 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:10.334326 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.335041 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.336221 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:10.336604 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:10.336698 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:10.337653 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:10.337662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:10.337676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:10.338257 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:10.338305 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:10.338325 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:10.338413 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:10.338515 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 07:44:10.338604 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:44:10.339434 29451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:10.390918 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.390374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50394)
0926 07:44:10.390603 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.390737 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 07:44:10.392103 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:10.391622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:41701 (local address 127.0.0.1:50396)
0926 07:44:10.391846 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:10.391932 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:41701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 07:44:10.427855 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29329
I20260926 07:44:10.444515 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 28978
I20260926 07:44:10.445767 29039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:10.554122 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 28978
2026-09-26T07:44:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:10 dist-test-slave-x492 krb5kdc[29510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:10 dist-test-slave-x492 krb5kdc[29510](info): set up 2 sockets
Sep 26 07:44:10 dist-test-slave-x492 krb5kdc[29510](info): commencing operation
krb5kdc: starting...
W20260926 07:44:11.657342 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:11 dist-test-slave-x492 krb5kdc[29510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:11.884027 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:33601
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:12.211203 29523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:12.211481 29523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:12.211587 29523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:12.218722 29523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:12.218832 29523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:12.218878 29523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:12.218927 29523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:12.218971 29523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:12.228713 29523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:12.229967 29523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:12.231218 29523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:12.242508 29528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:12.242570 29531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:12.242624 29529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:12.243728 29523 server_base.cc:1034] running on GCE node
I20260926 07:44:12.244472 29523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:12.245991 29523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:12.247218 29523 hybrid_clock.cc:648] HybridClock initialized: now 1790408652247152 us; error 49 us; skew 500 ppm
I20260926 07:44:12.247581 29523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:12 dist-test-slave-x492 krb5kdc[29510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408652, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:12.250707 29523 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:12.252125 29523 webserver.cc:460] Webserver started at http://127.11.9.126:33821/ using document root <none> and password file <none>
I20260926 07:44:12.252799 29523 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:12.252940 29523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:12.253288 29523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:12.256208 29523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "6ccd14b03a5243ad8cc18eaee8687dd4"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "66c47df4fa3bf2ebb2f8b9f63e7168ed"
server_key_iv: "125b532c727fa028fa68b62ab343c9ad"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.256943 29523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "6ccd14b03a5243ad8cc18eaee8687dd4"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "66c47df4fa3bf2ebb2f8b9f63e7168ed"
server_key_iv: "125b532c727fa028fa68b62ab343c9ad"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.262039 29523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260926 07:44:12.265336 29538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:12.266613 29523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:44:12.266791 29523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "6ccd14b03a5243ad8cc18eaee8687dd4"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "66c47df4fa3bf2ebb2f8b9f63e7168ed"
server_key_iv: "125b532c727fa028fa68b62ab343c9ad"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.266997 29523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:12.288601 29523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:12.292316 29523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:12.292594 29523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:12.309983 29523 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:33601
I20260926 07:44:12.310017 29589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:33601 every 8 connection(s)
I20260926 07:44:12.311280 29523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:12.314924 29590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:12.314945 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29523
I20260926 07:44:12.315184 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:12.315639 11301 external_mini_cluster.cc:1428] Setting key 4cee57ded011d8c198d293dc145b42c7
I20260926 07:44:12.329442 29590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: Bootstrap starting.
Sep 26 07:44:12 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:12.333199 29590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:12.334275 29590 log.cc:824] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:12.336970 29590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: No bootstrap required, opened a new log
I20260926 07:44:12.338521 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:12.317469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.11.9.126:33601)
0926 07:44:12.318334 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:12.318348 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:12.318388 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:44:12.319907 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:12.319925 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:12.320278 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:12.320602 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:12.322339 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.323589 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.325521 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.326209 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.326345 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:12.331546 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:12.331553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:12.331566 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:12.331603 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:12.334713 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:12.335378 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:12.335384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:12.335390 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:12.335589 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:12.335961 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:12.335964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:12.335967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:12.336446 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:12.336761 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:12.337182 (+ 421us) server_negotiation.cc:299] Negotiation successful
0926 07:44:12.337615 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":464,"threads_started":1}
I20260926 07:44:12.344590 29590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } }
I20260926 07:44:12.344952 29590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:12.345054 29590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ccd14b03a5243ad8cc18eaee8687dd4, State: Initialized, Role: FOLLOWER
I20260926 07:44:12.345683 29590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } }
I20260926 07:44:12.345923 29590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:12.346056 29590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:12.346230 29590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:12.348441 29590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } }
I20260926 07:44:12.348924 29590 leader_election.cc:302] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ccd14b03a5243ad8cc18eaee8687dd4; no voters:
I20260926 07:44:12.349572 29590 leader_election.cc:288] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:12.349713 29595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:12.350930 29595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [term 1 LEADER]: Becoming Leader. State: Replica: 6ccd14b03a5243ad8cc18eaee8687dd4, State: Running, Role: LEADER
I20260926 07:44:12.351444 29595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } }
I20260926 07:44:12.351734 29590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:12.354559 29596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } } }
I20260926 07:44:12.354717 29597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ccd14b03a5243ad8cc18eaee8687dd4. Latest consensus state: current_term: 1 leader_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccd14b03a5243ad8cc18eaee8687dd4" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 33601 } } }
I20260926 07:44:12.355252 29596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:12.355597 29597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:12.356586 29604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:12.362596 29604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:12.370728 29604 catalog_manager.cc:1347] Generated new cluster ID: da1107c961b54df3bb1613ff9e63ecfb
I20260926 07:44:12.370849 29604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:12.393292 29604 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:12.395251 29604 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:12.409878 29604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: Generated new TSK 0
I20260926 07:44:12.410889 29604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:44:12.474714 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:12.805756 29616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:12.806067 29616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:12.806226 29616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:12.813265 29616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:12.813400 29616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:12.813591 29616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:12.822952 29616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:12.824208 29616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:12.825544 29616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:12.838338 29622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:12.838600 29624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:12.838883 29621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:12.839872 29616 server_base.cc:1034] running on GCE node
I20260926 07:44:12.840476 29616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:12.841403 29616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:12.842615 29616 hybrid_clock.cc:648] HybridClock initialized: now 1790408652842567 us; error 44 us; skew 500 ppm
I20260926 07:44:12.842911 29616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:12 dist-test-slave-x492 krb5kdc[29510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408652, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:12.846014 29616 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:44:12.847344 29616 webserver.cc:460] Webserver started at http://127.11.9.65:34827/ using document root <none> and password file <none>
I20260926 07:44:12.848001 29616 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:12.848140 29616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:12.848452 29616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:12.851293 29616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "57c8d1743500440493c72aa105770f9e"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "169a50b279410cbaacba83f9eb3e0f28"
server_key_iv: "9cc57b6b5b7372021249193656e7dac4"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.852103 29616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "57c8d1743500440493c72aa105770f9e"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "169a50b279410cbaacba83f9eb3e0f28"
server_key_iv: "9cc57b6b5b7372021249193656e7dac4"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.857625 29616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 07:44:12.860857 29631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:12.862146 29616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:12.862318 29616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "57c8d1743500440493c72aa105770f9e"
format_stamp: "Formatted at 2026-09-26 07:44:12 on dist-test-slave-x492"
server_key: "169a50b279410cbaacba83f9eb3e0f28"
server_key_iv: "9cc57b6b5b7372021249193656e7dac4"
server_key_version: "encryptionkey@0"
I20260926 07:44:12.862504 29616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:12.902058 29616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:12.907294 29616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:12.907594 29616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:12.908771 29616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:12.910174 29616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:12.910248 29616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:12.910423 29616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:12.910481 29616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:12.943316 29616 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:36617
I20260926 07:44:12.943375 29743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:36617 every 8 connection(s)
I20260926 07:44:12.944619 29616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:12.950021 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29616
I20260926 07:44:12.950203 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:12.950737 11301 external_mini_cluster.cc:1428] Setting key 3cb07a98536b26908690a9d3c1142502
Sep 26 07:44:12 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408652, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:12.966670 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:12.947151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:54619 (local address 127.11.9.126:33601)
0926 07:44:12.947624 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:12.947628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:12.948502 (+ 874us) server_negotiation.cc:407] Connection header received
0926 07:44:12.950821 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:12.950830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:12.950921 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:12.951030 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:12.952945 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.953778 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.956300 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.956739 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.956841 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:12.960983 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:12.960987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:12.960991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:12.961022 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:12.963560 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:12.964273 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:12.964280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:12.964284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:12.964406 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:12.964812 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:12.964818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:12.964821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:12.965085 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:12.965261 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:12.966045 (+ 784us) server_negotiation.cc:299] Negotiation successful
0926 07:44:12.966290 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":254,"threads_started":1}
I20260926 07:44:12.967643 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:12.947667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:33601 (local address 127.11.9.65:54619)
0926 07:44:12.948344 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:12.948386 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:12.950474 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:12.951313 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:12.951321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:12.952020 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:12.952771 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:12.952795 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.953942 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.953951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:12.956156 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:12.956174 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:12.956953 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:12.956962 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:12.957312 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:12.958066 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:12.958100 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:12.960765 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:12.963703 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:12.963714 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:12.963730 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:12.964151 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:12.964514 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:12.964518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:12.964522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:12.964704 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:12.965249 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:12.965257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:12.965618 (+ 361us) client_negotiation.cc:769] Sending connection context
0926 07:44:12.965942 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 07:44:12.966464 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260926 07:44:12.969235 29744 heartbeater.cc:347] Connected to a master server at 127.11.9.126:33601
I20260926 07:44:12.969568 29744 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:12.970429 29744 heartbeater.cc:511] Master 127.11.9.126:33601 requested a full tablet report, sending...
I20260926 07:44:12.972875 29555 ts_manager.cc:194] Registered new tserver with Master: 57c8d1743500440493c72aa105770f9e (127.11.9.65:36617)
I20260926 07:44:12.975111 29555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:54619
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:44:13.006594 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:13.324813 29750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:13.325121 29750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:13.325335 29750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:13.332326 29750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:13.332450 29750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:13.332639 29750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:13.342056 29750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:13.343240 29750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:13.344476 29750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:13.357682 29758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:13.357600 29755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:13.357661 29756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:13.358392 29750 server_base.cc:1034] running on GCE node
I20260926 07:44:13.358908 29750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:13.359740 29750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:13.360925 29750 hybrid_clock.cc:648] HybridClock initialized: now 1790408653360879 us; error 42 us; skew 500 ppm
I20260926 07:44:13.361307 29750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:13 dist-test-slave-x492 krb5kdc[29510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408653, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:13.367619 29750 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:44:13.369076 29750 webserver.cc:460] Webserver started at http://127.11.9.66:40355/ using document root <none> and password file <none>
I20260926 07:44:13.369797 29750 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:13.369920 29750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:13.370231 29750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:13.373113 29750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "9cb656f8cc194f1d814f4e38ebd2d19a"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "f80b3511a3b821882c64b46e4aca8a41"
server_key_iv: "deb75a8f32834ad909f1866eb815ca3d"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.373898 29750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "9cb656f8cc194f1d814f4e38ebd2d19a"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "f80b3511a3b821882c64b46e4aca8a41"
server_key_iv: "deb75a8f32834ad909f1866eb815ca3d"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.379009 29750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:13.382402 29765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.383725 29750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:13.383888 29750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "9cb656f8cc194f1d814f4e38ebd2d19a"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "f80b3511a3b821882c64b46e4aca8a41"
server_key_iv: "deb75a8f32834ad909f1866eb815ca3d"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.384080 29750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:13.408356 29750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:13.412369 29750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:13.412683 29750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:13.413664 29750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:13.414983 29750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:13.415083 29750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.415226 29750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:13.415294 29750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.447832 29750 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:44771
I20260926 07:44:13.447876 29877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:44771 every 8 connection(s)
I20260926 07:44:13.449136 29750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:13.449499 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29750
I20260926 07:44:13.449695 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:13.450271 11301 external_mini_cluster.cc:1428] Setting key d2211f3b89920ba2064e9e4460e0a06b
Sep 26 07:44:13 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408653, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:13.469630 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:13.452105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:51777 (local address 127.11.9.126:33601)
0926 07:44:13.452324 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:13.452330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:13.453772 (+ 1442us) server_negotiation.cc:407] Connection header received
0926 07:44:13.455552 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:13.455558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:13.455644 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:13.455758 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:13.457351 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.458137 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.459627 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.460037 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.460167 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:13.463738 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:13.463743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:13.463746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:13.463776 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:13.466348 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:13.467108 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:13.467113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:13.467117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:13.467240 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:13.467654 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:13.467659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:13.467662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:13.467921 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:13.468090 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:13.468869 (+ 779us) server_negotiation.cc:299] Negotiation successful
0926 07:44:13.469143 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 07:44:13.470152 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:13.452807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:33601 (local address 127.11.9.66:51777)
0926 07:44:13.453635 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:13.453672 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:13.455294 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:13.455973 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:13.455981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:13.456560 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:13.457144 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:13.457162 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.458286 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.458295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:13.459474 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:13.459485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.460250 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.460257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:13.460479 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:13.461150 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:13.461220 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:13.463518 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:13.466469 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:13.466476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:13.466490 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:13.466987 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:13.467346 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:13.467351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:13.467355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:13.467534 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:13.468044 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:13.468051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:13.468375 (+ 324us) client_negotiation.cc:769] Sending connection context
0926 07:44:13.468693 (+ 318us) client_negotiation.cc:240] Negotiation successful
0926 07:44:13.469058 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":380,"threads_started":1}
I20260926 07:44:13.471591 29878 heartbeater.cc:347] Connected to a master server at 127.11.9.126:33601
I20260926 07:44:13.471920 29878 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:13.472587 29878 heartbeater.cc:511] Master 127.11.9.126:33601 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:44:13.474047 29555 ts_manager.cc:194] Registered new tserver with Master: 9cb656f8cc194f1d814f4e38ebd2d19a (127.11.9.66:44771)
I20260926 07:44:13.474830 29555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:51777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:44:13.503728 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:13.826375 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:13.826705 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:13.826866 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:13.834090 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:13.834240 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:13.834408 29883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:13.844084 29883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:13.845451 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:13.846752 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:13.859563 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:13.859603 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:13.859637 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:13.860378 29883 server_base.cc:1034] running on GCE node
I20260926 07:44:13.860952 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:13.861900 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:13.863116 29883 hybrid_clock.cc:648] HybridClock initialized: now 1790408653863061 us; error 48 us; skew 500 ppm
I20260926 07:44:13.863409 29883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:13 dist-test-slave-x492 krb5kdc[29510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408653, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:13.866618 29883 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:44:13.868029 29883 webserver.cc:460] Webserver started at http://127.11.9.67:41197/ using document root <none> and password file <none>
I20260926 07:44:13.868691 29883 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:13.868825 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:13.869120 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:13.872079 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "c515a6f506664b79af231d00efbf4893"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "d6835624730c8144344e2b27c8f29754"
server_key_iv: "2a74b2d720c9ef53b9efc88d45a19929"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.872840 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "c515a6f506664b79af231d00efbf4893"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "d6835624730c8144344e2b27c8f29754"
server_key_iv: "2a74b2d720c9ef53b9efc88d45a19929"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.877776 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 07:44:13.880898 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.882139 29883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 07:44:13.882303 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "c515a6f506664b79af231d00efbf4893"
format_stamp: "Formatted at 2026-09-26 07:44:13 on dist-test-slave-x492"
server_key: "d6835624730c8144344e2b27c8f29754"
server_key_iv: "2a74b2d720c9ef53b9efc88d45a19929"
server_key_version: "encryptionkey@0"
I20260926 07:44:13.882493 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:13.921659 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:13.925604 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:13.925881 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:13.926844 29883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:13.928143 29883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:13.928248 29883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.928391 29883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:13.928460 29883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:13.961364 29883 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:46309
I20260926 07:44:13.961447 30010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:46309 every 8 connection(s)
I20260926 07:44:13.962662 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:13.965886 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 29883
I20260926 07:44:13.966078 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:13.966646 11301 external_mini_cluster.cc:1428] Setting key fca97c0e5926ab6e1e64010de2d8bd7e
Sep 26 07:44:13 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408653, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:13.978479 29744 heartbeater.cc:503] Master 127.11.9.126:33601 was elected leader, sending a full tablet report...
I20260926 07:44:13.984935 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:13.965286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:55511 (local address 127.11.9.126:33601)
0926 07:44:13.965460 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:13.965464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:13.966808 (+ 1344us) server_negotiation.cc:407] Connection header received
0926 07:44:13.969138 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:13.969141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:13.969251 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:13.969329 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:13.971151 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.971964 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.973498 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.973892 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.973982 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:13.977681 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:13.977688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:13.977696 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:13.977746 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:13.981532 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:13.982435 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:13.982442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:13.982446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:13.982574 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:13.983116 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:13.983121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:13.983125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:13.983404 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:13.983572 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:13.984388 (+ 816us) server_negotiation.cc:299] Negotiation successful
0926 07:44:13.984596 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:44:13.986058 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:13.965753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:33601 (local address 127.11.9.67:55511)
0926 07:44:13.966611 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:13.966673 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:13.968868 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:13.969636 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:13.969648 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:13.970192 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:13.970785 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:13.970801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.972091 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.972095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:13.973384 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:13.973396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:13.974062 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:13.974066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:13.974265 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:13.974996 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:13.975018 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:13.977454 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:13.981656 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:13.981674 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:13.981692 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:13.982276 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:13.982711 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:13.982717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:13.982721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:13.982954 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:13.983533 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:13.983542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:13.983969 (+ 427us) client_negotiation.cc:769] Sending connection context
0926 07:44:13.984281 (+ 312us) client_negotiation.cc:240] Negotiation successful
0926 07:44:13.984811 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":355,"threads_started":1}
I20260926 07:44:13.987277 30011 heartbeater.cc:347] Connected to a master server at 127.11.9.126:33601
I20260926 07:44:13.987522 30011 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:13.988220 30011 heartbeater.cc:511] Master 127.11.9.126:33601 requested a full tablet report, sending...
I20260926 07:44:13.989617 29555 ts_manager.cc:194] Registered new tserver with Master: c515a6f506664b79af231d00efbf4893 (127.11.9.67:46309)
I20260926 07:44:13.990383 29555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:55511
I20260926 07:44:13.993923 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:14.010497 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:13.997822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.11.9.126:33601)
0926 07:44:13.997981 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:13.998001 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:13.998412 (+ 411us) server_negotiation.cc:407] Connection header received
0926 07:44:13.998610 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:13.998613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:13.998669 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:13.998766 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:14.000103 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.000797 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.002112 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.002526 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.002649 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.005386 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:14.005391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:14.005394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:14.005423 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:14.007698 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.008461 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.008466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.008475 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.008602 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.009033 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.009037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.009039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.009371 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:14.009659 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.010046 (+ 387us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.010220 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:44:14.014175 29555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:14.020875 29555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:14.059772 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.033242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.11.9.66:44771)
0926 07:44:14.047831 (+ 14589us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.047845 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.050845 (+ 3000us) server_negotiation.cc:407] Connection header received
0926 07:44:14.051318 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.051329 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.051641 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.051830 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:14.052869 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054437 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.057439 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.058770 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.058901 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.059100 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.059248 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.059433 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14354,"thread_start_us":14201,"threads_started":1}
I20260926 07:44:14.059829 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.032694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:44771 (local address 127.0.0.1:41272)
0926 07:44:14.049476 (+ 16782us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:14.049531 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:14.051037 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:14.052116 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:14.052120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:14.052147 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:14.052621 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.052631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054579 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.054585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.057301 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.057313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.058931 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.058936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.059062 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.059098 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:44:14.059208 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 07:44:14.059327 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16420,"mutex_wait_us":119,"spinlock_wait_cycles":5632,"thread_start_us":16207,"threads_started":1}
I20260926 07:44:14.060407 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.032782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36617 (local address 127.0.0.1:48500)
0926 07:44:14.050693 (+ 17911us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:14.050715 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:14.051024 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:14.051876 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:14.051883 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:14.051919 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:14.052412 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.052429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054616 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.054621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.057747 (+ 3126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.057761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.059553 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.059559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.059736 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.059779 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 07:44:14.059875 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 07:44:14.059989 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17752,"mutex_wait_us":425,"spinlock_wait_cycles":5632,"thread_start_us":17597,"threads_started":1}
I20260926 07:44:14.061053 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.032843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.11.9.65:36617)
0926 07:44:14.050220 (+ 17377us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.050233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.050824 (+ 591us) server_negotiation.cc:407] Connection header received
0926 07:44:14.051324 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.051337 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.051618 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.051840 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:14.052641 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054432 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.057902 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.059342 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.059464 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.059673 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.060704 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.060820 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17143,"thread_start_us":17761,"threads_started":1}
I20260926 07:44:14.062686 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.034308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.11.9.67:46309)
0926 07:44:14.051367 (+ 17059us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.051375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.052081 (+ 706us) server_negotiation.cc:407] Connection header received
0926 07:44:14.052168 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.052172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.052320 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.052499 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:14.053124 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054623 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.059762 (+ 5139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.061145 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.061314 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.061533 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.062016 (+ 483us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.062326 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16885,"thread_start_us":16729,"threads_started":1}
I20260926 07:44:14.064337 29813 tablet_service.cc:1467] Processing CreateTablet for tablet db1ef0793a0c42b2b2c873f400585b18 (DEFAULT_TABLE table=test-table [id=082d60e7c79849fcbfa05f5613ca1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:14.065871 29813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1ef0793a0c42b2b2c873f400585b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:14.065760 29679 tablet_service.cc:1467] Processing CreateTablet for tablet db1ef0793a0c42b2b2c873f400585b18 (DEFAULT_TABLE table=test-table [id=082d60e7c79849fcbfa05f5613ca1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:14.067173 29679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1ef0793a0c42b2b2c873f400585b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:14.069538 30027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.033643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:46309 (local address 127.0.0.1:56968)
0926 07:44:14.051828 (+ 18185us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:14.051845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:14.051952 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:14.052500 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:14.052505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:14.052527 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:14.052965 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.052974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.054767 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.054774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.057747 (+ 2973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.057761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.061664 (+ 3903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.061670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.061825 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.061855 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 07:44:14.061953 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 07:44:14.062080 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18037,"mutex_wait_us":546,"thread_start_us":17865,"threads_started":1}
I20260926 07:44:14.072526 29946 tablet_service.cc:1467] Processing CreateTablet for tablet db1ef0793a0c42b2b2c873f400585b18 (DEFAULT_TABLE table=test-table [id=082d60e7c79849fcbfa05f5613ca1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:14.073720 29946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1ef0793a0c42b2b2c873f400585b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:14.080716 30029 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Bootstrap starting.
I20260926 07:44:14.084411 30029 tablet_bootstrap.cc:654] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:14.086490 30029 log.cc:824] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:14.086882 30031 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Bootstrap starting.
I20260926 07:44:14.088007 30030 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Bootstrap starting.
I20260926 07:44:14.090432 30029 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: No bootstrap required, opened a new log
I20260926 07:44:14.090562 30031 tablet_bootstrap.cc:654] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:14.090728 30029 ts_tablet_manager.cc:1397] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260926 07:44:14.091449 30030 tablet_bootstrap.cc:654] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:14.091773 30031 log.cc:824] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:14.092639 30030 log.cc:824] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:14.094578 30031 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: No bootstrap required, opened a new log
I20260926 07:44:14.094898 30031 ts_tablet_manager.cc:1397] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20260926 07:44:14.095570 30030 tablet_bootstrap.cc:492] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: No bootstrap required, opened a new log
I20260926 07:44:14.095815 30030 ts_tablet_manager.cc:1397] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 07:44:14.098848 30029 raft_consensus.cc:348] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.099211 30029 raft_consensus.cc:374] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:14.099331 30029 raft_consensus.cc:729] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb656f8cc194f1d814f4e38ebd2d19a, State: Initialized, Role: FOLLOWER
I20260926 07:44:14.099913 30029 consensus_queue.cc:260] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.100973 29878 heartbeater.cc:503] Master 127.11.9.126:33601 was elected leader, sending a full tablet report...
I20260926 07:44:14.101670 30029 ts_tablet_manager.cc:1428] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260926 07:44:14.106320 30030 raft_consensus.cc:348] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.106745 30030 raft_consensus.cc:374] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:14.106853 30030 raft_consensus.cc:729] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57c8d1743500440493c72aa105770f9e, State: Initialized, Role: FOLLOWER
I20260926 07:44:14.106936 30031 raft_consensus.cc:348] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.107321 30031 raft_consensus.cc:374] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:14.107419 30031 raft_consensus.cc:729] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c515a6f506664b79af231d00efbf4893, State: Initialized, Role: FOLLOWER
I20260926 07:44:14.107553 30030 consensus_queue.cc:260] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.108004 30031 consensus_queue.cc:260] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.109081 30011 heartbeater.cc:503] Master 127.11.9.126:33601 was elected leader, sending a full tablet report...
I20260926 07:44:14.109289 30030 ts_tablet_manager.cc:1428] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260926 07:44:14.109658 30031 ts_tablet_manager.cc:1428] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
W20260926 07:44:14.196973 29745 tablet.cc:2369] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:14.201581 29879 tablet.cc:2369] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:14.214258 30012 tablet.cc:2369] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:14.317700 30035 raft_consensus.cc:482] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:14.318023 30035 raft_consensus.cc:504] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.319685 30035 leader_election.cc:288] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57c8d1743500440493c72aa105770f9e (127.11.9.65:36617), c515a6f506664b79af231d00efbf4893 (127.11.9.67:46309)
I20260926 07:44:14.329120 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.320159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:46309 (local address 127.11.9.66:40983)
0926 07:44:14.322253 (+ 2094us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:14.322269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:14.322448 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:14.322933 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:14.322937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:14.322959 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:14.323308 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.323318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.324543 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.324548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.326698 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.326713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.328305 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.328311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.328554 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.328600 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 07:44:14.328688 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 07:44:14.328820 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1977,"mutex_wait_us":30,"thread_start_us":1796,"threads_started":1}
I20260926 07:44:14.329146 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.321000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:40983 (local address 127.11.9.67:46309)
0926 07:44:14.321289 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.321297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.322352 (+ 1055us) server_negotiation.cc:407] Connection header received
0926 07:44:14.322588 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.322594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.322684 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.322744 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:14.323469 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.324414 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.326862 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.328098 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.328212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.328317 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.328746 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.328896 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 07:44:14.329247 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.320392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:37635 (local address 127.11.9.65:36617)
0926 07:44:14.320651 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.320660 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.322118 (+ 1458us) server_negotiation.cc:407] Connection header received
0926 07:44:14.322391 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.322395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.322451 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.322564 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:14.323055 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.324064 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.326870 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.328078 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.328212 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.328312 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.328886 (+ 574us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.329011 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 07:44:14.329890 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.319977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:36617 (local address 127.11.9.66:37635)
0926 07:44:14.321983 (+ 2006us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:14.322006 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:14.322235 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:14.322590 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:14.322594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:14.322619 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:14.322917 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.322925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.324206 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.324213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.326699 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:14.326713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.328502 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.328508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:14.328720 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.328751 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 07:44:14.329402 (+ 651us) client_negotiation.cc:240] Negotiation successful
0926 07:44:14.329553 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1803,"mutex_wait_us":95,"thread_start_us":1578,"threads_started":1}
I20260926 07:44:14.330188 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1ef0793a0c42b2b2c873f400585b18" candidate_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c515a6f506664b79af231d00efbf4893" is_pre_election: true
I20260926 07:44:14.330753 29966 raft_consensus.cc:2454] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb656f8cc194f1d814f4e38ebd2d19a in term 0.
I20260926 07:44:14.330878 29699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1ef0793a0c42b2b2c873f400585b18" candidate_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57c8d1743500440493c72aa105770f9e" is_pre_election: true
I20260926 07:44:14.331458 29699 raft_consensus.cc:2454] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb656f8cc194f1d814f4e38ebd2d19a in term 0.
I20260926 07:44:14.331470 29769 leader_election.cc:302] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb656f8cc194f1d814f4e38ebd2d19a, c515a6f506664b79af231d00efbf4893; no voters:
I20260926 07:44:14.331868 30035 raft_consensus.cc:2781] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:14.332057 30035 raft_consensus.cc:482] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:14.332191 30035 raft_consensus.cc:3037] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:14.334381 30035 raft_consensus.cc:504] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.334937 30035 leader_election.cc:288] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [CANDIDATE]: Term 1 election: Requested vote from peers 57c8d1743500440493c72aa105770f9e (127.11.9.65:36617), c515a6f506664b79af231d00efbf4893 (127.11.9.67:46309)
I20260926 07:44:14.335309 29699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1ef0793a0c42b2b2c873f400585b18" candidate_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57c8d1743500440493c72aa105770f9e"
I20260926 07:44:14.335382 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1ef0793a0c42b2b2c873f400585b18" candidate_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c515a6f506664b79af231d00efbf4893"
I20260926 07:44:14.335498 29699 raft_consensus.cc:3037] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:14.335605 29966 raft_consensus.cc:3037] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:14.337769 29699 raft_consensus.cc:2454] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb656f8cc194f1d814f4e38ebd2d19a in term 1.
I20260926 07:44:14.337864 29966 raft_consensus.cc:2454] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb656f8cc194f1d814f4e38ebd2d19a in term 1.
I20260926 07:44:14.338176 29767 leader_election.cc:302] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57c8d1743500440493c72aa105770f9e, 9cb656f8cc194f1d814f4e38ebd2d19a; no voters:
I20260926 07:44:14.338466 30035 raft_consensus.cc:2781] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:14.338872 30035 raft_consensus.cc:686] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 1 LEADER]: Becoming Leader. State: Replica: 9cb656f8cc194f1d814f4e38ebd2d19a, State: Running, Role: LEADER
I20260926 07:44:14.339381 30035 consensus_queue.cc:237] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } }
I20260926 07:44:14.343649 29554 catalog_manager.cc:5495] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb656f8cc194f1d814f4e38ebd2d19a (127.11.9.66). New cstate: current_term: 1 leader_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb656f8cc194f1d814f4e38ebd2d19a" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 44771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 } health_report { overall_health: UNKNOWN } } }
I20260926 07:44:14.410561 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.405678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.11.9.66:44771)
0926 07:44:14.405860 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.405865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.405930 (+ 65us) server_negotiation.cc:407] Connection header received
0926 07:44:14.406113 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.406117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.406208 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.406333 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:14.406649 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.407364 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.408616 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.408985 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.409075 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.409613 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:14.410087 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:14.410226 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.410315 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.410354 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:44:14.418371 29966 raft_consensus.cc:1261] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 1 FOLLOWER]: Refusing update from remote peer 9cb656f8cc194f1d814f4e38ebd2d19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:14.418398 29699 raft_consensus.cc:1261] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 1 FOLLOWER]: Refusing update from remote peer 9cb656f8cc194f1d814f4e38ebd2d19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:14.419301 30040 consensus_queue.cc:1035] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57c8d1743500440493c72aa105770f9e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 36617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:14.419687 30035 consensus_queue.cc:1035] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c515a6f506664b79af231d00efbf4893" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:14.445611 29555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:14.445930 29555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:14.449838 29555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ccd14b03a5243ad8cc18eaee8687dd4: Sending DeleteTablet for 3 replicas of tablet db1ef0793a0c42b2b2c873f400585b18
I20260926 07:44:14.450875 29679 tablet_service.cc:1514] Processing DeleteTablet for tablet db1ef0793a0c42b2b2c873f400585b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:14 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:48500
I20260926 07:44:14.451220 29946 tablet_service.cc:1514] Processing DeleteTablet for tablet db1ef0793a0c42b2b2c873f400585b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:14 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:56968
I20260926 07:44:14.455598 29813 tablet_service.cc:1514] Processing DeleteTablet for tablet db1ef0793a0c42b2b2c873f400585b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:14 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:41272
I20260926 07:44:14.458566 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29616
I20260926 07:44:14.477785 30055 tablet_replica.cc:330] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: stopping tablet replica
I20260926 07:44:14.478243 30055 raft_consensus.cc:2229] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:44:14.478941 30055 raft_consensus.cc:2258] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:14.480324 30054 tablet_replica.cc:330] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: stopping tablet replica
I20260926 07:44:14.480734 30054 raft_consensus.cc:2229] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:14.481264 30054 raft_consensus.cc:2258] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:14.482052 30053 tablet_replica.cc:330] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: stopping tablet replica
I20260926 07:44:14.482463 30053 raft_consensus.cc:2229] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:14.482980 30053 raft_consensus.cc:2258] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:14.484683 30055 ts_tablet_manager.cc:1905] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:14.485234 30053 ts_tablet_manager.cc:1905] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:14.488293 30054 ts_tablet_manager.cc:1905] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 07:44:14 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:44:14.492902 30053 ts_tablet_manager.cc:1918] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:14.493012 30055 ts_tablet_manager.cc:1918] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:14.493212 30055 log.cc:1196] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/db1ef0793a0c42b2b2c873f400585b18
I20260926 07:44:14.493257 30053 log.cc:1196] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/db1ef0793a0c42b2b2c873f400585b18
I20260926 07:44:14.493724 30055 ts_tablet_manager.cc:1939] T db1ef0793a0c42b2b2c873f400585b18 P 9cb656f8cc194f1d814f4e38ebd2d19a: Deleting consensus metadata
I20260926 07:44:14.494056 30053 ts_tablet_manager.cc:1939] T db1ef0793a0c42b2b2c873f400585b18 P 57c8d1743500440493c72aa105770f9e: Deleting consensus metadata
I20260926 07:44:14.495185 30054 ts_tablet_manager.cc:1918] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:14.495327 30054 log.cc:1196] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/db1ef0793a0c42b2b2c873f400585b18
I20260926 07:44:14.495851 30054 ts_tablet_manager.cc:1939] T db1ef0793a0c42b2b2c873f400585b18 P c515a6f506664b79af231d00efbf4893: Deleting consensus metadata
I20260926 07:44:14.495795 29539 catalog_manager.cc:4841] TS 9cb656f8cc194f1d814f4e38ebd2d19a (127.11.9.66:44771): tablet db1ef0793a0c42b2b2c873f400585b18 (table test-table [id=082d60e7c79849fcbfa05f5613ca1705]) successfully deleted
I20260926 07:44:14.496428 29540 catalog_manager.cc:4841] TS 57c8d1743500440493c72aa105770f9e (127.11.9.65:36617): tablet db1ef0793a0c42b2b2c873f400585b18 (table test-table [id=082d60e7c79849fcbfa05f5613ca1705]) successfully deleted
I20260926 07:44:14.497310 29542 catalog_manager.cc:4841] TS c515a6f506664b79af231d00efbf4893 (127.11.9.67:46309): tablet db1ef0793a0c42b2b2c873f400585b18 (table test-table [id=082d60e7c79849fcbfa05f5613ca1705]) successfully deleted
I20260926 07:44:14.498898 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.461530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.11.9.65:36617)
0926 07:44:14.461756 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.461766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.479696 (+ 17930us) server_negotiation.cc:407] Connection header received
0926 07:44:14.479896 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.479905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.480067 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.480204 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:14.482360 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.483304 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.485414 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.486054 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.486154 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.491898 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:14.491912 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:14.491929 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:14.491966 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:14.495917 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.496708 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.496717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.496720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.496858 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.497325 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.497331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.497334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.497728 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:14.498043 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.498330 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.498540 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:44:14.499598 29739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:14.614279 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29616
I20260926 07:44:14.638137 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29750
Sep 26 07:44:14 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:44:14.653878 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.638984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.11.9.66:44771)
0926 07:44:14.639154 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.639158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.639175 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:44:14.639319 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.639323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.639390 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.639475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:14.641076 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.641921 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.643228 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.643651 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.643804 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.648049 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:14.648062 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:14.648077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:14.648107 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:14.651137 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.651809 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.651817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.651819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.651957 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.652346 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.652352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.652355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.652696 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:14.653048 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.653288 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.653506 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 07:44:14.654536 29873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:14.768214 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29750
I20260926 07:44:14.787041 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29883
Sep 26 07:44:14 dist-test-slave-x492 krb5kdc[29510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:44:14.801865 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:14.787791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.11.9.67:46309)
0926 07:44:14.787966 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:14.787971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:14.787991 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:44:14.788107 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:14.788110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:14.788174 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:14.788270 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:14.789552 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.790467 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.791686 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:14.792055 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:14.792151 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:14.796636 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:14.796649 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:14.796665 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:14.796696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:14.799569 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.800161 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.800168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.800170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.800301 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:14.800601 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:14.800604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:14.800605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:14.800972 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:14.801251 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:14.801469 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 07:44:14.801620 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:44:14.802484 30006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:14.890627 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29883
I20260926 07:44:14.907387 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 29523
I20260926 07:44:14.908449 29585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:15.023155 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 29523
2026-09-26T07:44:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4470 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T07:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:15Z Disabled control of system clock
I20260926 07:44:15.097110 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37799
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:37799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:15.420483 30064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:15.420751 30064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:15.420831 30064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:15.429132 30064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:15.429270 30064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:15.429365 30064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:15.429483 30064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:15.429566 30064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:15.439878 30064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:37799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:15.441242 30064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:15.443161 30064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:15.455631 30070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:15.455569 30069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:15.455665 30072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:15.456528 30064 server_base.cc:1034] running on GCE node
I20260926 07:44:15.457330 30064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:15.458621 30064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:15.459838 30064 hybrid_clock.cc:648] HybridClock initialized: now 1790408655459767 us; error 68 us; skew 500 ppm
I20260926 07:44:15.460189 30064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:15.462416 30064 webserver.cc:460] Webserver started at http://127.11.9.126:45987/ using document root <none> and password file <none>
I20260926 07:44:15.463084 30064 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:15.463209 30064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:15.463513 30064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:15.466349 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "e42556d5c50d42ad9b17cce6d24efefe"
format_stamp: "Formatted at 2026-09-26 07:44:15 on dist-test-slave-x492"
server_key: "293abeb4fd08ecfd95440cde4a61c6b9"
server_key_iv: "ae6a78d23cead2c79e38e3878c3b8a80"
server_key_version: "encryptionkey@0"
I20260926 07:44:15.467123 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "e42556d5c50d42ad9b17cce6d24efefe"
format_stamp: "Formatted at 2026-09-26 07:44:15 on dist-test-slave-x492"
server_key: "293abeb4fd08ecfd95440cde4a61c6b9"
server_key_iv: "ae6a78d23cead2c79e38e3878c3b8a80"
server_key_version: "encryptionkey@0"
I20260926 07:44:15.471894 30064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 07:44:15.475178 30078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:15.476416 30064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 07:44:15.476594 30064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "e42556d5c50d42ad9b17cce6d24efefe"
format_stamp: "Formatted at 2026-09-26 07:44:15 on dist-test-slave-x492"
server_key: "293abeb4fd08ecfd95440cde4a61c6b9"
server_key_iv: "ae6a78d23cead2c79e38e3878c3b8a80"
server_key_version: "encryptionkey@0"
I20260926 07:44:15.476794 30064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:15.517427 30064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:15.518358 30064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:15.518658 30064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:15.537050 30064 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:37799
I20260926 07:44:15.537026 30129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:37799 every 8 connection(s)
I20260926 07:44:15.538414 30064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:15.542096 30130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:15.548296 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30064
I20260926 07:44:15.548513 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:15.549058 11301 external_mini_cluster.cc:1428] Setting key 0310949ed722c6d7bf6e26f4604bec93
I20260926 07:44:15.552990 30130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: Bootstrap starting.
I20260926 07:44:15.556262 30130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:15.557538 30130 log.cc:824] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:15.560644 30130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: No bootstrap required, opened a new log
I20260926 07:44:15.562995 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:15.551076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.11.9.126:37799)
0926 07:44:15.552022 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:15.552032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:15.552073 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:44:15.553894 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:15.553917 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:15.554383 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:15.554719 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:15.555291 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:15.556587 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:15.558496 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:15.559279 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:15.559426 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:15.560636 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:15.560644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:15.560656 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:15.561313 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:15.561394 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:15.561420 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:15.561505 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:15.561729 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 07:44:15.562039 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":336,"threads_started":1}
I20260926 07:44:15.568257 30130 raft_consensus.cc:348] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } }
I20260926 07:44:15.568609 30130 raft_consensus.cc:374] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:15.568727 30130 raft_consensus.cc:729] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e42556d5c50d42ad9b17cce6d24efefe, State: Initialized, Role: FOLLOWER
I20260926 07:44:15.569331 30130 consensus_queue.cc:260] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } }
I20260926 07:44:15.569562 30130 raft_consensus.cc:388] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:15.569710 30130 raft_consensus.cc:482] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:15.569870 30130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:15.572029 30130 raft_consensus.cc:504] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } }
I20260926 07:44:15.572506 30130 leader_election.cc:302] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e42556d5c50d42ad9b17cce6d24efefe; no voters:
I20260926 07:44:15.573055 30130 leader_election.cc:288] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:15.573225 30135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:15.573908 30135 raft_consensus.cc:686] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [term 1 LEADER]: Becoming Leader. State: Replica: e42556d5c50d42ad9b17cce6d24efefe, State: Running, Role: LEADER
I20260926 07:44:15.574438 30135 consensus_queue.cc:237] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } }
I20260926 07:44:15.575179 30130 sys_catalog.cc:564] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:15.577001 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e42556d5c50d42ad9b17cce6d24efefe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } } }
I20260926 07:44:15.577421 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:15.578909 30141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:15.578958 30137 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [sys.catalog]: SysCatalogTable state changed. Reason: New leader e42556d5c50d42ad9b17cce6d24efefe. Latest consensus state: current_term: 1 leader_uuid: "e42556d5c50d42ad9b17cce6d24efefe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42556d5c50d42ad9b17cce6d24efefe" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37799 } } }
I20260926 07:44:15.579242 30137 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:15.583524 30141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:15.593389 30141 catalog_manager.cc:1347] Generated new cluster ID: b822f39733c944d68a1891a819e05762
I20260926 07:44:15.593506 30141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:15.620833 30141 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:15.622207 30141 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:15.632790 30141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: Generated new TSK 0
I20260926 07:44:15.633685 30141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:15.642784 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--builtin_ntp_servers=127.11.9.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:15.982122 30154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:15.982489 30154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:15.982673 30154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:15.991340 30154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:15.991475 30154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:15.991644 30154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:16.000761 30154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:16.002128 30154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:16.003901 30154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:16.017553 30160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.017635 30162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.017669 30159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:16.018404 30154 server_base.cc:1034] running on GCE node
I20260926 07:44:16.018953 30154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:16.019841 30154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:16.021049 30154 hybrid_clock.cc:648] HybridClock initialized: now 1790408656021008 us; error 28 us; skew 500 ppm
I20260926 07:44:16.021401 30154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:16.023721 30154 webserver.cc:460] Webserver started at http://127.11.9.65:37403/ using document root <none> and password file <none>
I20260926 07:44:16.024389 30154 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:16.024502 30154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:16.024832 30154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:16.027679 30154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "5d43945da4f54c14beee295689d540f9"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "c7eef62c2832646d11ff8575ae275e80"
server_key_iv: "1c858c467a9ac5efa4b28a689d31ede1"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.028467 30154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "5d43945da4f54c14beee295689d540f9"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "c7eef62c2832646d11ff8575ae275e80"
server_key_iv: "1c858c467a9ac5efa4b28a689d31ede1"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.033499 30154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 07:44:16.036749 30168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.038050 30154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:16.038208 30154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "5d43945da4f54c14beee295689d540f9"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "c7eef62c2832646d11ff8575ae275e80"
server_key_iv: "1c858c467a9ac5efa4b28a689d31ede1"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.038399 30154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:16.080298 30154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:16.081212 30154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:16.081502 30154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:16.082515 30154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:16.083957 30154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:16.084069 30154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.084199 30154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:16.084302 30154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.117645 30154 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:43061
I20260926 07:44:16.117688 30280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:43061 every 8 connection(s)
I20260926 07:44:16.118976 30154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:16.119865 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30154
I20260926 07:44:16.120031 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:16.120620 11301 external_mini_cluster.cc:1428] Setting key edc4dc0602184e473bd5af5f840d74aa
I20260926 07:44:16.123739 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--builtin_ntp_servers=127.11.9.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:16.131532 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:16.121828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:49281 (local address 127.11.9.126:37799)
0926 07:44:16.122342 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:16.122349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:16.123432 (+ 1083us) server_negotiation.cc:407] Connection header received
0926 07:44:16.125267 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:16.125272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:16.125362 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:16.125497 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:16.126497 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.127222 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.128773 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.129230 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.129369 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:16.130792 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:16.130797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:16.130801 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:16.130819 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:16.130860 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:16.130872 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:16.130948 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:16.131207 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 07:44:16.131286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":278,"threads_started":1}
I20260926 07:44:16.133033 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:16.122320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37799 (local address 127.11.9.65:49281)
0926 07:44:16.123290 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:16.123330 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:16.124996 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:16.125712 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:16.125721 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:16.125809 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:16.126334 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:16.126349 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.127379 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.127384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:16.128625 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:16.128636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.129457 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.129465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:16.129687 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:16.130414 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:16.130420 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:16.130474 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:16.130584 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:16.130976 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:16.130987 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:16.130995 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 07:44:16.131180 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 07:44:16.131589 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":323,"threads_started":1}
I20260926 07:44:16.134506 30281 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37799
I20260926 07:44:16.134836 30281 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:16.135692 30281 heartbeater.cc:511] Master 127.11.9.126:37799 requested a full tablet report, sending...
I20260926 07:44:16.137869 30095 ts_manager.cc:194] Registered new tserver with Master: 5d43945da4f54c14beee295689d540f9 (127.11.9.65:43061)
I20260926 07:44:16.139276 30095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.65:49281
W20260926 07:44:16.443537 30285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:16.443799 30285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:16.443954 30285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:16.451597 30285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:16.451686 30285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:16.451839 30285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:16.460803 30285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:16.462134 30285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:16.463941 30285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:16.477116 30291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.477269 30290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.477216 30293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:16.478317 30285 server_base.cc:1034] running on GCE node
I20260926 07:44:16.478962 30285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:16.479985 30285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:16.481338 30285 hybrid_clock.cc:648] HybridClock initialized: now 1790408656481283 us; error 39 us; skew 500 ppm
I20260926 07:44:16.481660 30285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:16.483917 30285 webserver.cc:460] Webserver started at http://127.11.9.66:43867/ using document root <none> and password file <none>
I20260926 07:44:16.484592 30285 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:16.484706 30285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:16.485016 30285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:16.488056 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "d3528b44522245d6b01f466159f84ea3"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "8f8b7f58144496d7fd138caa16147d75"
server_key_iv: "1f0c3a902d4a7a55760914d83e3fe581"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.488970 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "d3528b44522245d6b01f466159f84ea3"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "8f8b7f58144496d7fd138caa16147d75"
server_key_iv: "1f0c3a902d4a7a55760914d83e3fe581"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.493960 30285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 07:44:16.497334 30299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.498641 30285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:44:16.498802 30285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "d3528b44522245d6b01f466159f84ea3"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "8f8b7f58144496d7fd138caa16147d75"
server_key_iv: "1f0c3a902d4a7a55760914d83e3fe581"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.499017 30285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:16.517324 30285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:16.518162 30285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:16.518433 30285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:16.519387 30285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:16.520787 30285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:16.520870 30285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.521019 30285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:16.521124 30285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.554008 30285 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:35061
I20260926 07:44:16.554041 30411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:35061 every 8 connection(s)
I20260926 07:44:16.555634 30285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:16.558918 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30285
I20260926 07:44:16.559113 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:16.559718 11301 external_mini_cluster.cc:1428] Setting key a5a155723e6ebcfdd739a6803c3e575f
I20260926 07:44:16.562829 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--builtin_ntp_servers=127.11.9.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:16.570614 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:16.558411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:36675 (local address 127.11.9.126:37799)
0926 07:44:16.558629 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:16.558635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:16.560465 (+ 1830us) server_negotiation.cc:407] Connection header received
0926 07:44:16.562510 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:16.562516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:16.562592 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:16.562692 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:16.564096 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.564786 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.567231 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.567851 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.567974 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:16.569702 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:16.569710 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:16.569717 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:16.569741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:16.569796 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:16.569813 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:16.569908 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:16.570162 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 07:44:16.570266 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 07:44:16.571774 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:16.558996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37799 (local address 127.11.9.66:36675)
0926 07:44:16.560308 (+ 1312us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:16.560346 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:16.562205 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:16.562963 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:16.562977 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:16.563097 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:16.563918 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:16.563944 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.564940 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.564948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:16.567092 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:16.567110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:16.568078 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:16.568088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:16.568435 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:16.569301 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:16.569307 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:16.569366 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:16.569485 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:16.569931 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:16.569944 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:16.569954 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 07:44:16.570123 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 07:44:16.570546 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":350,"threads_started":1}
I20260926 07:44:16.573179 30412 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37799
I20260926 07:44:16.573499 30412 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:16.574225 30412 heartbeater.cc:511] Master 127.11.9.126:37799 requested a full tablet report, sending...
I20260926 07:44:16.575796 30095 ts_manager.cc:194] Registered new tserver with Master: d3528b44522245d6b01f466159f84ea3 (127.11.9.66:35061)
I20260926 07:44:16.576644 30095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.66:36675
W20260926 07:44:16.883787 30415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:16.884042 30415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:16.884204 30415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:16.891599 30415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:16.891693 30415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:16.891880 30415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:16.900810 30415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:16.902165 30415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:16.904039 30415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:16.922431 30420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.922390 30421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:16.922309 30423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:16.923192 30415 server_base.cc:1034] running on GCE node
I20260926 07:44:16.923743 30415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:16.924629 30415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:16.925853 30415 hybrid_clock.cc:648] HybridClock initialized: now 1790408656925800 us; error 65 us; skew 500 ppm
I20260926 07:44:16.926179 30415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:16.928289 30415 webserver.cc:460] Webserver started at http://127.11.9.67:41883/ using document root <none> and password file <none>
I20260926 07:44:16.928993 30415 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:16.929142 30415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:16.929495 30415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:16.932271 30415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "6ed8c58235a24a4bbf8ab4d184c147ff"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "ca1f0ef5a45b9e57974a3e8ed8601e7f"
server_key_iv: "b46531c97d89b261c5c7a6d5d5ea7308"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.933043 30415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "6ed8c58235a24a4bbf8ab4d184c147ff"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "ca1f0ef5a45b9e57974a3e8ed8601e7f"
server_key_iv: "b46531c97d89b261c5c7a6d5d5ea7308"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.938024 30415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 07:44:16.941500 30429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.942965 30415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:44:16.943135 30415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "6ed8c58235a24a4bbf8ab4d184c147ff"
format_stamp: "Formatted at 2026-09-26 07:44:16 on dist-test-slave-x492"
server_key: "ca1f0ef5a45b9e57974a3e8ed8601e7f"
server_key_iv: "b46531c97d89b261c5c7a6d5d5ea7308"
server_key_version: "encryptionkey@0"
I20260926 07:44:16.943347 30415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:16.971060 30415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:16.971943 30415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:16.972234 30415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:16.973218 30415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:16.974923 30415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:16.975032 30415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:16.975157 30415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:16.975262 30415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:17.018199 30415 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:38817
I20260926 07:44:17.018286 30541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:38817 every 8 connection(s)
I20260926 07:44:17.019515 30415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:17.020459 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30415
I20260926 07:44:17.020641 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:17.021277 11301 external_mini_cluster.cc:1428] Setting key e03524df8e71b47dbd6014a4f24a3455
I20260926 07:44:17.032070 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.022413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:46851 (local address 127.11.9.126:37799)
0926 07:44:17.022605 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.022611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.024416 (+ 1805us) server_negotiation.cc:407] Connection header received
0926 07:44:17.026180 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.026184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.026250 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.026331 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:17.027271 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.027978 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.029593 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.029976 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.030076 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.031335 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:17.031339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:17.031342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:17.031359 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:17.031389 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:17.031426 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:17.031536 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.031733 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.031807 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:44:17.033054 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.023041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:37799 (local address 127.11.9.67:46851)
0926 07:44:17.024225 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.024273 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.025948 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.026493 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.026500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.026585 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:17.027128 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.027151 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.028129 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.028132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.029475 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.029487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.030177 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.030182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.030378 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.031022 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:17.031025 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:17.031064 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:17.031155 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:17.031539 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:17.031547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:17.031554 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.031709 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.032006 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":365,"threads_started":1}
I20260926 07:44:17.034601 30542 heartbeater.cc:347] Connected to a master server at 127.11.9.126:37799
I20260926 07:44:17.034904 30542 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:17.035596 30542 heartbeater.cc:511] Master 127.11.9.126:37799 requested a full tablet report, sending...
I20260926 07:44:17.037003 30095 ts_manager.cc:194] Registered new tserver with Master: 6ed8c58235a24a4bbf8ab4d184c147ff (127.11.9.67:38817)
I20260926 07:44:17.037921 30095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.67:46851
I20260926 07:44:17.048977 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:17.059391 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.053467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.11.9.126:37799)
0926 07:44:17.053630 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.053635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.054123 (+ 488us) server_negotiation.cc:407] Connection header received
0926 07:44:17.054261 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.054265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.054330 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.054423 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:17.054852 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.055657 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.056963 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.057547 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.057651 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.058722 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:17.058727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:17.058731 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:17.058748 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:17.058778 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:17.058790 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:17.058956 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.059104 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.059151 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:44:17.063216 30095 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:17.070048 30095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:17.103406 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.082446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:35061 (local address 127.0.0.1:60722)
0926 07:44:17.095742 (+ 13296us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.095760 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.096601 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.097295 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.097303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.097330 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:17.097715 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.097727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.099136 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.099146 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.101094 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.101106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.102433 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.102439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.102611 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.102664 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.102796 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.102976 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13150,"mutex_wait_us":2,"thread_start_us":376,"threads_started":1}
I20260926 07:44:17.103405 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.082782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.11.9.66:35061)
0926 07:44:17.090079 (+ 7297us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.090093 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.095860 (+ 5767us) server_negotiation.cc:407] Connection header received
0926 07:44:17.096793 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.096801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.097076 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.097280 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:17.097889 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.099002 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.101318 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.102248 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.102420 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.102710 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.102839 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.103042 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":802,"threads_started":1}
I20260926 07:44:17.103698 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.082059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:43061 (local address 127.0.0.1:41660)
0926 07:44:17.083148 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.083183 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.083335 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.095986 (+ 12651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.095993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.096028 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:17.096604 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.096618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.098419 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.098425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.101011 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.101026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.102906 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.102912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.103068 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.103100 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.103162 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.103295 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":97,"thread_start_us":14286,"threads_started":1}
I20260926 07:44:17.104833 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.082285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.11.9.65:43061)
0926 07:44:17.095105 (+ 12820us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.095119 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.095165 (+ 46us) server_negotiation.cc:407] Connection header received
0926 07:44:17.095304 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.095316 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.095670 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.095932 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:17.096815 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.098280 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.101264 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.102696 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.104039 (+ 1343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.104226 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.104301 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.104475 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12572,"thread_start_us":13727,"threads_started":1}
I20260926 07:44:17.105186 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.083577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38817 (local address 127.0.0.1:57806)
0926 07:44:17.096695 (+ 13118us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.096713 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.096820 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.097652 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.097659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.097693 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:17.098048 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.098056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.099906 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.099911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.102082 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.102096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.104413 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.104419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.104585 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.104621 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.104707 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.104834 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12963,"mutex_wait_us":42,"thread_start_us":12282,"threads_started":1}
I20260926 07:44:17.107671 30347 tablet_service.cc:1467] Processing CreateTablet for tablet 9c568685cada42b69e4bfea16c13ad85 (DEFAULT_TABLE table=test-table [id=eee7f14b262044fd9ebce044e8077ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:17.105693 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.084158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.11.9.67:38817)
0926 07:44:17.095140 (+ 10982us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.095149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.096837 (+ 1688us) server_negotiation.cc:407] Connection header received
0926 07:44:17.096963 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.096968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.097129 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.097354 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:17.098218 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.099712 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.102285 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.104212 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.104348 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.104596 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.104830 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.105139 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10790,"thread_start_us":10980,"threads_started":1}
I20260926 07:44:17.108314 30216 tablet_service.cc:1467] Processing CreateTablet for tablet 9c568685cada42b69e4bfea16c13ad85 (DEFAULT_TABLE table=test-table [id=eee7f14b262044fd9ebce044e8077ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:17.109176 30347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c568685cada42b69e4bfea16c13ad85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:17.109674 30216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c568685cada42b69e4bfea16c13ad85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:17.111449 30477 tablet_service.cc:1467] Processing CreateTablet for tablet 9c568685cada42b69e4bfea16c13ad85 (DEFAULT_TABLE table=test-table [id=eee7f14b262044fd9ebce044e8077ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:17.112490 30477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c568685cada42b69e4bfea16c13ad85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:17.130559 30560 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Bootstrap starting.
I20260926 07:44:17.132947 30561 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Bootstrap starting.
I20260926 07:44:17.136917 30560 tablet_bootstrap.cc:654] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:17.137291 30561 tablet_bootstrap.cc:654] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:17.138296 30560 log.cc:824] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:17.138573 30562 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Bootstrap starting.
I20260926 07:44:17.138631 30561 log.cc:824] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:17.142025 30561 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: No bootstrap required, opened a new log
I20260926 07:44:17.142282 30561 ts_tablet_manager.cc:1397] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 07:44:17.142606 30562 tablet_bootstrap.cc:654] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:17.142791 30281 heartbeater.cc:503] Master 127.11.9.126:37799 was elected leader, sending a full tablet report...
I20260926 07:44:17.143751 30562 log.cc:824] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:17.149878 30562 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: No bootstrap required, opened a new log
I20260926 07:44:17.150154 30562 ts_tablet_manager.cc:1397] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260926 07:44:17.152285 30561 raft_consensus.cc:348] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.152670 30561 raft_consensus.cc:374] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:17.152772 30561 raft_consensus.cc:729] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed8c58235a24a4bbf8ab4d184c147ff, State: Initialized, Role: FOLLOWER
I20260926 07:44:17.153434 30561 consensus_queue.cc:260] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.154467 30542 heartbeater.cc:503] Master 127.11.9.126:37799 was elected leader, sending a full tablet report...
I20260926 07:44:17.155753 30561 ts_tablet_manager.cc:1428] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 07:44:17.156437 30560 tablet_bootstrap.cc:492] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: No bootstrap required, opened a new log
I20260926 07:44:17.156731 30560 ts_tablet_manager.cc:1397] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.000s
I20260926 07:44:17.161664 30562 raft_consensus.cc:348] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.162161 30562 raft_consensus.cc:374] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:17.162281 30562 raft_consensus.cc:729] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3528b44522245d6b01f466159f84ea3, State: Initialized, Role: FOLLOWER
I20260926 07:44:17.162840 30562 consensus_queue.cc:260] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.163928 30412 heartbeater.cc:503] Master 127.11.9.126:37799 was elected leader, sending a full tablet report...
I20260926 07:44:17.164533 30562 ts_tablet_manager.cc:1428] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260926 07:44:17.168377 30560 raft_consensus.cc:348] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.168829 30560 raft_consensus.cc:374] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:17.168947 30560 raft_consensus.cc:729] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d43945da4f54c14beee295689d540f9, State: Initialized, Role: FOLLOWER
I20260926 07:44:17.169739 30560 consensus_queue.cc:260] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.171623 30560 ts_tablet_manager.cc:1428] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260926 07:44:17.271339 30543 tablet.cc:2369] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:17.303369 30568 raft_consensus.cc:482] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:17.303711 30568 raft_consensus.cc:504] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.305555 30568 leader_election.cc:288] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3528b44522245d6b01f466159f84ea3 (127.11.9.66:35061), 6ed8c58235a24a4bbf8ab4d184c147ff (127.11.9.67:38817)
W20260926 07:44:17.307724 30413 tablet.cc:2369] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:17.313364 30569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.305930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:35061 (local address 127.11.9.65:45789)
0926 07:44:17.306616 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.306637 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.306838 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.307316 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.307320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.307345 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:17.307751 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.307763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.308939 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.308942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.311082 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.311099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.312699 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.312703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.312817 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.312840 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.312913 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.312998 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":25,"thread_start_us":375,"threads_started":1}
I20260926 07:44:17.313385 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.305939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:45789 (local address 127.11.9.66:35061)
0926 07:44:17.306168 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.306177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.306731 (+ 554us) server_negotiation.cc:407] Connection header received
0926 07:44:17.306988 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.306995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.307101 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.307211 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:17.307942 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.308812 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.311330 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.312541 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.312670 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.312765 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.312972 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.313100 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 07:44:17.314018 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.306984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:49109 (local address 127.11.9.67:38817)
0926 07:44:17.307214 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.307219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.307533 (+ 314us) server_negotiation.cc:407] Connection header received
0926 07:44:17.307711 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.307718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.307794 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.307913 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:17.308580 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.309443 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.311798 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.313037 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.313185 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.313281 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.313652 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.313765 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260926 07:44:17.314340 30367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c568685cada42b69e4bfea16c13ad85" candidate_uuid: "5d43945da4f54c14beee295689d540f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3528b44522245d6b01f466159f84ea3" is_pre_election: true
I20260926 07:44:17.314684 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.306652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:38817 (local address 127.11.9.65:49109)
0926 07:44:17.307408 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:17.307428 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:17.307569 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:17.307994 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:17.307999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:17.308023 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:17.308390 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.308403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.309590 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.309595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.311597 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:17.311611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.313247 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.313253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:17.313456 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.313492 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 07:44:17.314217 (+ 725us) client_negotiation.cc:240] Negotiation successful
0926 07:44:17.314346 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":1,"thread_start_us":445,"threads_started":1}
I20260926 07:44:17.314965 30367 raft_consensus.cc:2454] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d43945da4f54c14beee295689d540f9 in term 0.
I20260926 07:44:17.315676 30172 leader_election.cc:302] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d43945da4f54c14beee295689d540f9, d3528b44522245d6b01f466159f84ea3; no voters:
I20260926 07:44:17.315687 30497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c568685cada42b69e4bfea16c13ad85" candidate_uuid: "5d43945da4f54c14beee295689d540f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" is_pre_election: true
I20260926 07:44:17.316109 30568 raft_consensus.cc:2781] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:17.316255 30568 raft_consensus.cc:482] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:17.316176 30497 raft_consensus.cc:2454] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d43945da4f54c14beee295689d540f9 in term 0.
I20260926 07:44:17.316355 30568 raft_consensus.cc:3037] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:17.318666 30568 raft_consensus.cc:504] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.319238 30568 leader_election.cc:288] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [CANDIDATE]: Term 1 election: Requested vote from peers d3528b44522245d6b01f466159f84ea3 (127.11.9.66:35061), 6ed8c58235a24a4bbf8ab4d184c147ff (127.11.9.67:38817)
I20260926 07:44:17.319764 30367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c568685cada42b69e4bfea16c13ad85" candidate_uuid: "5d43945da4f54c14beee295689d540f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3528b44522245d6b01f466159f84ea3"
I20260926 07:44:17.319846 30497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c568685cada42b69e4bfea16c13ad85" candidate_uuid: "5d43945da4f54c14beee295689d540f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff"
I20260926 07:44:17.319979 30367 raft_consensus.cc:3037] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:17.320029 30497 raft_consensus.cc:3037] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:17.323199 30367 raft_consensus.cc:2454] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d43945da4f54c14beee295689d540f9 in term 1.
I20260926 07:44:17.323199 30497 raft_consensus.cc:2454] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d43945da4f54c14beee295689d540f9 in term 1.
I20260926 07:44:17.323555 30172 leader_election.cc:302] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d43945da4f54c14beee295689d540f9, d3528b44522245d6b01f466159f84ea3; no voters:
I20260926 07:44:17.323827 30568 raft_consensus.cc:2781] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:17.324306 30568 raft_consensus.cc:686] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 1 LEADER]: Becoming Leader. State: Replica: 5d43945da4f54c14beee295689d540f9, State: Running, Role: LEADER
I20260926 07:44:17.324761 30568 consensus_queue.cc:237] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } }
I20260926 07:44:17.329480 30095 catalog_manager.cc:5495] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d43945da4f54c14beee295689d540f9 (127.11.9.65). New cstate: current_term: 1 leader_uuid: "5d43945da4f54c14beee295689d540f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d43945da4f54c14beee295689d540f9" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 43061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 } health_report { overall_health: UNKNOWN } } }
W20260926 07:44:17.371417 30282 tablet.cc:2369] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:17.396922 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.391558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41664 (local address 127.11.9.65:43061)
0926 07:44:17.391742 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.391747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.391766 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:44:17.391870 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.391874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.391963 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.392096 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:17.392417 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.393221 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.394691 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.395212 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.395333 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.395781 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:17.396324 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:17.396481 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.396623 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.396662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 07:44:17.404956 30367 raft_consensus.cc:1261] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 1 FOLLOWER]: Refusing update from remote peer 5d43945da4f54c14beee295689d540f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:17.405021 30497 raft_consensus.cc:1261] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 1 FOLLOWER]: Refusing update from remote peer 5d43945da4f54c14beee295689d540f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:17.405908 30571 consensus_queue.cc:1035] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3528b44522245d6b01f466159f84ea3" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:17.406287 30568 consensus_queue.cc:1035] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed8c58235a24a4bbf8ab4d184c147ff" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:17.428893 30094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:17.429224 30094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:17.432881 30094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e42556d5c50d42ad9b17cce6d24efefe: Sending DeleteTablet for 3 replicas of tablet 9c568685cada42b69e4bfea16c13ad85
I20260926 07:44:17.434011 30216 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c568685cada42b69e4bfea16c13ad85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:17 UTC) from {username='slave'} at 127.0.0.1:41660
I20260926 07:44:17.434726 30347 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c568685cada42b69e4bfea16c13ad85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:17 UTC) from {username='slave'} at 127.0.0.1:60722
I20260926 07:44:17.435477 30477 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c568685cada42b69e4bfea16c13ad85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:17 UTC) from {username='slave'} at 127.0.0.1:57806
I20260926 07:44:17.435577 30584 tablet_replica.cc:330] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: stopping tablet replica
I20260926 07:44:17.435750 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30154
I20260926 07:44:17.436045 30584 raft_consensus.cc:2229] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:44:17.437402 30584 raft_consensus.cc:2258] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:17.438266 30586 tablet_replica.cc:330] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: stopping tablet replica
I20260926 07:44:17.438745 30586 raft_consensus.cc:2229] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:17.439182 30586 raft_consensus.cc:2258] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:17.439195 30585 tablet_replica.cc:330] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: stopping tablet replica
I20260926 07:44:17.439631 30585 raft_consensus.cc:2229] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:17.440162 30585 raft_consensus.cc:2258] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:17.441852 30586 ts_tablet_manager.cc:1905] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:17.442528 30584 ts_tablet_manager.cc:1905] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:17.443954 30585 ts_tablet_manager.cc:1905] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:17.449178 30586 ts_tablet_manager.cc:1918] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:17.449390 30586 log.cc:1196] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/9c568685cada42b69e4bfea16c13ad85
I20260926 07:44:17.449999 30586 ts_tablet_manager.cc:1939] T 9c568685cada42b69e4bfea16c13ad85 P 6ed8c58235a24a4bbf8ab4d184c147ff: Deleting consensus metadata
I20260926 07:44:17.450523 30584 ts_tablet_manager.cc:1918] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:17.450690 30584 log.cc:1196] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/9c568685cada42b69e4bfea16c13ad85
I20260926 07:44:17.451195 30584 ts_tablet_manager.cc:1939] T 9c568685cada42b69e4bfea16c13ad85 P 5d43945da4f54c14beee295689d540f9: Deleting consensus metadata
I20260926 07:44:17.452076 30082 catalog_manager.cc:4841] TS 6ed8c58235a24a4bbf8ab4d184c147ff (127.11.9.67:38817): tablet 9c568685cada42b69e4bfea16c13ad85 (table test-table [id=eee7f14b262044fd9ebce044e8077ba9]) successfully deleted
I20260926 07:44:17.452793 30081 catalog_manager.cc:4841] TS 5d43945da4f54c14beee295689d540f9 (127.11.9.65:43061): tablet 9c568685cada42b69e4bfea16c13ad85 (table test-table [id=eee7f14b262044fd9ebce044e8077ba9]) successfully deleted
I20260926 07:44:17.452999 30585 ts_tablet_manager.cc:1918] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:17.453153 30585 log.cc:1196] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/9c568685cada42b69e4bfea16c13ad85
I20260926 07:44:17.453933 30585 ts_tablet_manager.cc:1939] T 9c568685cada42b69e4bfea16c13ad85 P d3528b44522245d6b01f466159f84ea3: Deleting consensus metadata
I20260926 07:44:17.455382 30082 catalog_manager.cc:4841] TS d3528b44522245d6b01f466159f84ea3 (127.11.9.66:35061): tablet 9c568685cada42b69e4bfea16c13ad85 (table test-table [id=eee7f14b262044fd9ebce044e8077ba9]) successfully deleted
I20260926 07:44:17.460569 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.436870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.11.9.65:43061)
0926 07:44:17.437059 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.437064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.453123 (+ 16059us) server_negotiation.cc:407] Connection header received
0926 07:44:17.453442 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.453447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.453576 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.453660 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:17.454408 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.455656 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.456899 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.457586 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.457728 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.458915 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:17.458925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:17.458941 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:17.459736 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:17.459797 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:17.459828 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:17.459950 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.460083 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.460178 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:44:17.461448 30276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:17.568918 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30154
I20260926 07:44:17.587878 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30285
I20260926 07:44:17.594542 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.588663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.11.9.66:35061)
0926 07:44:17.588836 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.588840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.588857 (+ 17us) server_negotiation.cc:407] Connection header received
0926 07:44:17.589004 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.589008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.589101 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.589251 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:17.589598 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.590423 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.591702 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.592079 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.592165 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.593117 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:17.593125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:17.593139 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:17.593813 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:17.593857 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:17.593876 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:17.593965 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.594107 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.594185 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:44:17.595244 30407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:17.683826 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30285
I20260926 07:44:17.700604 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30415
I20260926 07:44:17.707650 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:17.701416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.11.9.67:38817)
0926 07:44:17.701612 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:17.701618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:17.701692 (+ 74us) server_negotiation.cc:407] Connection header received
0926 07:44:17.701923 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:17.701928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:17.702033 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:17.702143 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:17.702546 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.703288 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.704580 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:17.704977 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:17.705114 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:17.706285 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:17.706293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:17.706309 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:17.706863 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:17.706910 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:17.706942 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:17.707092 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:17.707233 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 07:44:17.707338 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:44:17.708279 30537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:17.802729 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30415
I20260926 07:44:17.819245 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30064
I20260926 07:44:17.820420 30125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:17.934000 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30064
2026-09-26T07:44:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2909 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:18 dist-test-slave-x492 krb5kdc[30594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:18 dist-test-slave-x492 krb5kdc[30594](info): set up 2 sockets
Sep 26 07:44:18 dist-test-slave-x492 krb5kdc[30594](info): commencing operation
krb5kdc: starting...
W20260926 07:44:19.041579 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:19 dist-test-slave-x492 krb5kdc[30594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:19.281018 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45061
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:19.610682 30607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:19.611019 30607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:19.611176 30607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:19.618377 30607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:19.618515 30607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:19.618616 30607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:19.618711 30607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:19.618795 30607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:19.628331 30607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:19.629601 30607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:19.630911 30607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:19.642130 30613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:19.642179 30615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:19.642216 30612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:19.642987 30607 server_base.cc:1034] running on GCE node
I20260926 07:44:19.643689 30607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:19.644991 30607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:19.646207 30607 hybrid_clock.cc:648] HybridClock initialized: now 1790408659646179 us; error 52 us; skew 500 ppm
I20260926 07:44:19.646538 30607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:19 dist-test-slave-x492 krb5kdc[30594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:19.653124 30607 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:19.654587 30607 webserver.cc:460] Webserver started at http://127.11.9.126:39579/ using document root <none> and password file <none>
I20260926 07:44:19.655248 30607 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:19.655396 30607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:19.655690 30607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:19.658663 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "ac2d38a92b454594878e4cc61a31ec81"
format_stamp: "Formatted at 2026-09-26 07:44:19 on dist-test-slave-x492"
server_key: "ed719c1cab9605d855a739549604c6e7"
server_key_iv: "eab88ac7e71225eefa4db57107750c8e"
server_key_version: "encryptionkey@0"
I20260926 07:44:19.659430 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "ac2d38a92b454594878e4cc61a31ec81"
format_stamp: "Formatted at 2026-09-26 07:44:19 on dist-test-slave-x492"
server_key: "ed719c1cab9605d855a739549604c6e7"
server_key_iv: "eab88ac7e71225eefa4db57107750c8e"
server_key_version: "encryptionkey@0"
I20260926 07:44:19.664510 30607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 07:44:19.667847 30622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:19.669128 30607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:44:19.669492 30607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "ac2d38a92b454594878e4cc61a31ec81"
format_stamp: "Formatted at 2026-09-26 07:44:19 on dist-test-slave-x492"
server_key: "ed719c1cab9605d855a739549604c6e7"
server_key_iv: "eab88ac7e71225eefa4db57107750c8e"
server_key_version: "encryptionkey@0"
I20260926 07:44:19.669687 30607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:19.681661 30607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:19.685784 30607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:19.686071 30607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:19.704607 30607 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45061
I20260926 07:44:19.704641 30673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45061 every 8 connection(s)
I20260926 07:44:19.706076 30607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:19.709693 30674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:19.713758 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30607
I20260926 07:44:19.713959 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:19.714483 11301 external_mini_cluster.cc:1428] Setting key c75bb63681bc2ff27f8d137ebc2eeccd
I20260926 07:44:19.722851 30674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: Bootstrap starting.
I20260926 07:44:19.726596 30674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:19.727902 30674 log.cc:824] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: Log is configured to *not* fsync() on all Append() calls
Sep 26 07:44:19 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:19.730769 30674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: No bootstrap required, opened a new log
I20260926 07:44:19.738675 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:19.716396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.11.9.126:45061)
0926 07:44:19.717129 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:19.717139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:19.717225 (+ 86us) server_negotiation.cc:407] Connection header received
0926 07:44:19.718825 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:19.718846 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:19.719239 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:19.719563 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:19.721360 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:19.722629 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:19.724372 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:19.725034 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:19.725157 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:19.730181 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:19.730188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:19.730202 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:19.730264 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:19.734272 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:19.735020 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:19.735028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:19.735035 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:19.735250 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:19.735589 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:19.735595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:19.735599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:19.736248 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:19.736574 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:19.736863 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 07:44:19.737316 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":351,"threads_started":1}
I20260926 07:44:19.740933 30674 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } }
I20260926 07:44:19.741387 30674 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:19.741523 30674 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac2d38a92b454594878e4cc61a31ec81, State: Initialized, Role: FOLLOWER
I20260926 07:44:19.742041 30674 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } }
I20260926 07:44:19.742274 30674 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:19.742420 30674 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:19.742606 30674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:19.744673 30674 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } }
I20260926 07:44:19.745200 30674 leader_election.cc:302] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac2d38a92b454594878e4cc61a31ec81; no voters:
I20260926 07:44:19.745808 30674 leader_election.cc:288] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:19.745961 30679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:19.746695 30679 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [term 1 LEADER]: Becoming Leader. State: Replica: ac2d38a92b454594878e4cc61a31ec81, State: Running, Role: LEADER
I20260926 07:44:19.747309 30679 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } }
I20260926 07:44:19.748011 30674 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:19.750464 30680 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac2d38a92b454594878e4cc61a31ec81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } } }
I20260926 07:44:19.750459 30681 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac2d38a92b454594878e4cc61a31ec81. Latest consensus state: current_term: 1 leader_uuid: "ac2d38a92b454594878e4cc61a31ec81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2d38a92b454594878e4cc61a31ec81" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45061 } } }
I20260926 07:44:19.750881 30680 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:19.750891 30681 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:19.752791 30687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:19.758211 30687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:19.766556 30687 catalog_manager.cc:1347] Generated new cluster ID: 434ddaea85f441a98e1be0f6dfb69afc
I20260926 07:44:19.766669 30687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:19.796516 30687 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:19.797921 30687 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:19.808063 30687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: Generated new TSK 0
I20260926 07:44:19.809082 30687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.9.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
Entry for principal kudu/127.11.9.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab.
I20260926 07:44:19.873142 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:20.205210 30700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:20.205484 30700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:20.205578 30700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:20.212623 30700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:20.212726 30700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:20.212889 30700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:20.222815 30700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.65.keytab
--principal=kudu/127.11.9.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:20.224037 30700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:20.225332 30700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:20.239295 30708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:20.239519 30705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:20.240180 30706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:20.241444 30700 server_base.cc:1034] running on GCE node
I20260926 07:44:20.241963 30700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:20.242828 30700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:20.244035 30700 hybrid_clock.cc:648] HybridClock initialized: now 1790408660243986 us; error 37 us; skew 500 ppm
I20260926 07:44:20.244293 30700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:20 dist-test-slave-x492 krb5kdc[30594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408660, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:20.247337 30700 init.cc:377] Logged in from keytab as kudu/127.11.9.65@KRBTEST.COM (short username kudu)
I20260926 07:44:20.248838 30700 webserver.cc:460] Webserver started at http://127.11.9.65:33249/ using document root <none> and password file <none>
I20260926 07:44:20.249599 30700 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:20.249722 30700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:20.250041 30700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:20.253003 30700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "f45cf78713364102b8a1c431a322dd76"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "ed3538601f4551dadd76583077f683b0"
server_key_iv: "4ff2b617c36963633bbc498a16610e90"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.254043 30700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "f45cf78713364102b8a1c431a322dd76"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "ed3538601f4551dadd76583077f683b0"
server_key_iv: "4ff2b617c36963633bbc498a16610e90"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.259011 30700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:20.262378 30715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.263639 30700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:20.263794 30700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "f45cf78713364102b8a1c431a322dd76"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "ed3538601f4551dadd76583077f683b0"
server_key_iv: "4ff2b617c36963633bbc498a16610e90"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.264009 30700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:20.275200 30700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:20.279088 30700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:20.279361 30700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:20.280241 30700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:20.281526 30700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:20.281598 30700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.281744 30700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:20.281803 30700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.314671 30700 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:46863
I20260926 07:44:20.314716 30827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:46863 every 8 connection(s)
I20260926 07:44:20.315994 30700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:20.318390 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30700
I20260926 07:44:20.318585 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:20.319211 11301 external_mini_cluster.cc:1428] Setting key c71f124a356f7bf0f75c721a5ddca99a
Sep 26 07:44:20 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408660, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.65@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:20.336922 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:20.318684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:47815 (local address 127.11.9.126:45061)
0926 07:44:20.319167 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:20.319174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:20.320223 (+ 1049us) server_negotiation.cc:407] Connection header received
0926 07:44:20.322135 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:20.322144 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:20.322235 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:20.322365 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:20.323816 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.324932 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.326560 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.327334 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.327481 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:20.331166 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:20.331172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:20.331175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:20.331207 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:20.333927 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:20.334645 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:20.334655 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:20.334657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:20.334785 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:20.335214 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:20.335221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:20.335229 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:20.335483 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:20.335637 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:20.336341 (+ 704us) server_negotiation.cc:299] Negotiation successful
0926 07:44:20.336560 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":217,"threads_started":1}
I20260926 07:44:20.337893 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:20.319251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45061 (local address 127.11.9.65:47815)
0926 07:44:20.320063 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:20.320114 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:20.321872 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:20.322560 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:20.322567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:20.323099 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:20.323647 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:20.323664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.325089 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.325093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:20.326401 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:20.326416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.327544 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.327548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:20.327739 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:20.328407 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:20.328429 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:20.330951 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:20.334069 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:20.334078 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:20.334094 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:20.334515 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:20.334893 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:20.334899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:20.334902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:20.335094 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:20.335612 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:20.335620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:20.335946 (+ 326us) client_negotiation.cc:769] Sending connection context
0926 07:44:20.336215 (+ 269us) client_negotiation.cc:240] Negotiation successful
0926 07:44:20.336724 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":335,"threads_started":1}
I20260926 07:44:20.339504 30828 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45061
I20260926 07:44:20.339823 30828 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:20.340510 30828 heartbeater.cc:511] Master 127.11.9.126:45061 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.66@KRBTEST.COM; defaulting to no policy
I20260926 07:44:20.343003 30639 ts_manager.cc:194] Registered new tserver with Master: f45cf78713364102b8a1c431a322dd76 (127.11.9.65:46863)
I20260926 07:44:20.344563 30639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.65@KRBTEST.COM'} at 127.11.9.65:47815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
Entry for principal kudu/127.11.9.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab.
I20260926 07:44:20.375918 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:20.697691 30834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:20.697952 30834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:20.698102 30834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:20.705353 30834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:20.705456 30834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:20.705629 30834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:20.715091 30834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.66.keytab
--principal=kudu/127.11.9.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:20.716271 30834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:20.717586 30834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:20.730773 30840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:20.730855 30842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:20.730803 30839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:20.731475 30834 server_base.cc:1034] running on GCE node
I20260926 07:44:20.732060 30834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:20.732961 30834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:20.734185 30834 hybrid_clock.cc:648] HybridClock initialized: now 1790408660734128 us; error 70 us; skew 500 ppm
I20260926 07:44:20.734498 30834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:20 dist-test-slave-x492 krb5kdc[30594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408660, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:20.737610 30834 init.cc:377] Logged in from keytab as kudu/127.11.9.66@KRBTEST.COM (short username kudu)
I20260926 07:44:20.739101 30834 webserver.cc:460] Webserver started at http://127.11.9.66:45081/ using document root <none> and password file <none>
I20260926 07:44:20.739754 30834 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:20.739898 30834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:20.740201 30834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:20.743052 30834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "60e23973e4ae42ee880d75ab173ebc8f"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "fd38fbd8833e79603ec71e545c82382c"
server_key_iv: "47ccb61ecf889defd0fa2a72908fcf5f"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.743810 30834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "60e23973e4ae42ee880d75ab173ebc8f"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "fd38fbd8833e79603ec71e545c82382c"
server_key_iv: "47ccb61ecf889defd0fa2a72908fcf5f"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.749091 30834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 07:44:20.752511 30849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.753974 30834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 07:44:20.754133 30834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "60e23973e4ae42ee880d75ab173ebc8f"
format_stamp: "Formatted at 2026-09-26 07:44:20 on dist-test-slave-x492"
server_key: "fd38fbd8833e79603ec71e545c82382c"
server_key_iv: "47ccb61ecf889defd0fa2a72908fcf5f"
server_key_version: "encryptionkey@0"
I20260926 07:44:20.754338 30834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:20.766353 30834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:20.770526 30834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:20.770807 30834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:20.771715 30834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:20.773010 30834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:20.773111 30834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.773304 30834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:20.773381 30834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:20.806607 30834 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:35009
I20260926 07:44:20.806661 30961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:35009 every 8 connection(s)
I20260926 07:44:20.807909 30834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:20.809027 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30834
I20260926 07:44:20.809232 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:20.809732 11301 external_mini_cluster.cc:1428] Setting key d712d1f2a914534a14ed347e76a81206
Sep 26 07:44:20 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408660, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.66@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:20.829320 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:20.810701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:41267 (local address 127.11.9.126:45061)
0926 07:44:20.810927 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:20.810933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:20.812342 (+ 1409us) server_negotiation.cc:407] Connection header received
0926 07:44:20.814227 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:20.814235 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:20.814335 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:20.814445 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:20.815951 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.817144 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.818710 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.819350 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.819489 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:20.823515 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:20.823520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:20.823523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:20.823558 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:20.826231 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:20.826961 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:20.826967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:20.826970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:20.827099 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:20.827549 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:20.827555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:20.827558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:20.827852 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:20.828009 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:20.828696 (+ 687us) server_negotiation.cc:299] Negotiation successful
0926 07:44:20.828949 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:44:20.830307 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:20.811272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45061 (local address 127.11.9.66:41267)
0926 07:44:20.812181 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:20.812225 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:20.813971 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:20.814634 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:20.814642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:20.815221 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:20.815769 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:20.815786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.817302 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.817307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:20.818561 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:20.818577 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:20.819589 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:20.819597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:20.819833 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:20.820521 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:20.820555 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:20.823269 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:20.826382 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:20.826391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:20.826408 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:20.826837 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:20.827213 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:20.827218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:20.827222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:20.827429 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:20.827981 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:20.827988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:20.828333 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 07:44:20.828609 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 07:44:20.829204 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":401,"threads_started":1}
I20260926 07:44:20.831849 30962 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45061
I20260926 07:44:20.832216 30962 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:20.833113 30962 heartbeater.cc:511] Master 127.11.9.126:45061 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.9.67@KRBTEST.COM; defaulting to no policy
I20260926 07:44:20.834762 30639 ts_manager.cc:194] Registered new tserver with Master: 60e23973e4ae42ee880d75ab173ebc8f (127.11.9.66:35009)
I20260926 07:44:20.835602 30639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.66@KRBTEST.COM'} at 127.11.9.66:41267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
Entry for principal kudu/127.11.9.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab.
I20260926 07:44:20.864746 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:21.193840 30967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:21.194178 30967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:21.194355 30967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:21.201749 30967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:21.201915 30967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:21.202105 30967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:21.212121 30967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.67.keytab
--principal=kudu/127.11.9.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:45061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:21.213568 30967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:21.214910 30967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:21.228299 30972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:21.228327 30973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:21.228374 30975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:21.229511 30967 server_base.cc:1034] running on GCE node
I20260926 07:44:21.230093 30967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:21.231024 30967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:21.232223 30967 hybrid_clock.cc:648] HybridClock initialized: now 1790408661232182 us; error 28 us; skew 500 ppm
I20260926 07:44:21.232558 30967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:21 dist-test-slave-x492 krb5kdc[30594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408661, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:21.235764 30967 init.cc:377] Logged in from keytab as kudu/127.11.9.67@KRBTEST.COM (short username kudu)
I20260926 07:44:21.237322 30967 webserver.cc:460] Webserver started at http://127.11.9.67:39439/ using document root <none> and password file <none>
I20260926 07:44:21.238026 30967 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:21.238165 30967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:21.238483 30967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:21.241359 30967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "5f4e53efb1b743b089beb4d48cebcb3d"
format_stamp: "Formatted at 2026-09-26 07:44:21 on dist-test-slave-x492"
server_key: "64d6218ca9eed909ed982b12be718aae"
server_key_iv: "0c3c3e23db7bdc63f5630bb33c7386e6"
server_key_version: "encryptionkey@0"
I20260926 07:44:21.242111 30967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "5f4e53efb1b743b089beb4d48cebcb3d"
format_stamp: "Formatted at 2026-09-26 07:44:21 on dist-test-slave-x492"
server_key: "64d6218ca9eed909ed982b12be718aae"
server_key_iv: "0c3c3e23db7bdc63f5630bb33c7386e6"
server_key_version: "encryptionkey@0"
I20260926 07:44:21.247117 30967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:21.250514 30982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:21.251819 30967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:21.251981 30967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "5f4e53efb1b743b089beb4d48cebcb3d"
format_stamp: "Formatted at 2026-09-26 07:44:21 on dist-test-slave-x492"
server_key: "64d6218ca9eed909ed982b12be718aae"
server_key_iv: "0c3c3e23db7bdc63f5630bb33c7386e6"
server_key_version: "encryptionkey@0"
I20260926 07:44:21.252210 30967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:21.275166 30967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:21.279193 30967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:21.279484 30967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:21.280452 30967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:21.281816 30967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:21.281888 30967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:21.282023 30967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:21.282073 30967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:21.315412 30967 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:35347
I20260926 07:44:21.315441 31094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:35347 every 8 connection(s)
I20260926 07:44:21.316737 30967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:21.318617 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 30967
I20260926 07:44:21.318807 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:21.319308 11301 external_mini_cluster.cc:1428] Setting key 4efc0ba683c4f323c7b20138945ba084
Sep 26 07:44:21 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408661, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.67@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:21.336570 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.319458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:55677 (local address 127.11.9.126:45061)
0926 07:44:21.319673 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.319679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.321216 (+ 1537us) server_negotiation.cc:407] Connection header received
0926 07:44:21.323183 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.323187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.323253 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.323335 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:21.324700 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.325590 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.327025 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.327407 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.327498 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.330644 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:21.330649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:21.330652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:21.330682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:21.333529 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.334295 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.334301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.334306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.334436 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.334876 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.334882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.334885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.335151 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:21.335312 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.335997 (+ 685us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.336211 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 07:44:21.337534 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.320094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45061 (local address 127.11.9.67:55677)
0926 07:44:21.321051 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.321091 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.322948 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.323524 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.323531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.324037 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:21.324562 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.324578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.325724 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.325727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.326902 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.326913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.327575 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.327579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.327754 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.328358 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:44:21.328378 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:21.330471 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:21.333669 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:21.333677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:21.333692 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:21.334150 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:21.334548 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:44:21.334554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:44:21.334558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:44:21.334753 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:44:21.335295 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:21.335303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:21.335614 (+ 311us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.335904 (+ 290us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.336410 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":406,"threads_started":1}
I20260926 07:44:21.338744 31095 heartbeater.cc:347] Connected to a master server at 127.11.9.126:45061
I20260926 07:44:21.339054 31095 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:21.339717 31095 heartbeater.cc:511] Master 127.11.9.126:45061 requested a full tablet report, sending...
I20260926 07:44:21.341039 30639 ts_manager.cc:194] Registered new tserver with Master: 5f4e53efb1b743b089beb4d48cebcb3d (127.11.9.67:35347)
I20260926 07:44:21.341861 30639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.9.67@KRBTEST.COM'} at 127.11.9.67:55677
I20260926 07:44:21.346647 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:21.348268 30828 heartbeater.cc:503] Master 127.11.9.126:45061 was elected leader, sending a full tablet report...
I20260926 07:44:21.363909 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.350938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.11.9.126:45061)
0926 07:44:21.351128 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.351136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.351545 (+ 409us) server_negotiation.cc:407] Connection header received
0926 07:44:21.351736 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.351741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.351822 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.351923 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:21.353539 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.354362 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.355745 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.356166 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.356301 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.358697 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:21.358703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:21.358707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:21.358747 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:21.361245 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.361893 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.361900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.361903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.362041 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.362433 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.362439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.362443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.362750 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:21.363030 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.363359 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.363576 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 07:44:21.367501 30639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:21.374744 30639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:21.399376 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.387530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35347 (local address 127.0.0.1:60192)
0926 07:44:21.389862 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.389879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.390072 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.391021 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.391025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.391050 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:21.391628 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.391642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.393586 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.393591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.396090 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.396106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.398245 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.398251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.398519 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.398576 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.398709 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.398907 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1927,"mutex_wait_us":57,"thread_start_us":1800,"threads_started":1}
I20260926 07:44:21.400280 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.387686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60192 (local address 127.11.9.67:35347)
0926 07:44:21.389184 (+ 1498us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.389195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.389965 (+ 770us) server_negotiation.cc:407] Connection header received
0926 07:44:21.390259 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.390267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.390511 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.390676 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:21.391846 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.393432 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.396273 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.397720 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.397859 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.398118 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.399794 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.399966 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":1163,"threads_started":1}
I20260926 07:44:21.400635 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.386989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:35009 (local address 127.0.0.1:59550)
0926 07:44:21.388194 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.388253 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.388462 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.390856 (+ 2394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.390863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.390891 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:21.391401 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.391416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.394689 (+ 3273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.394694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.396811 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.396825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.399825 (+ 3000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.399831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.400003 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.400035 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.400101 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.400222 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":24,"thread_start_us":812,"threads_started":1}
I20260926 07:44:21.401638 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.387157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.11.9.66:35009)
0926 07:44:21.389466 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.389474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.389503 (+ 29us) server_negotiation.cc:407] Connection header received
0926 07:44:21.389600 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.389607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.390627 (+ 1020us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.391611 (+ 984us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:21.391846 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.394525 (+ 2679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.397010 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.399602 (+ 2592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.400772 (+ 1170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.400989 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.401059 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.401277 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2146,"thread_start_us":2026,"threads_started":1}
I20260926 07:44:21.405562 30897 tablet_service.cc:1467] Processing CreateTablet for tablet b75f832453784e4d9e1e9f0affdb732e (DEFAULT_TABLE table=test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:21.405562 31030 tablet_service.cc:1467] Processing CreateTablet for tablet b75f832453784e4d9e1e9f0affdb732e (DEFAULT_TABLE table=test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:21.407020 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75f832453784e4d9e1e9f0affdb732e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:21.407023 30897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75f832453784e4d9e1e9f0affdb732e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:21.407907 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.389650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:46863 (local address 127.0.0.1:49270)
0926 07:44:21.400971 (+ 11321us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.400990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.401084 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.402763 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.402779 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.402806 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:21.402942 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.402949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.404338 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.404346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.405857 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.405869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.407157 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.407162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.407354 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.407386 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.407504 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.407633 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11241,"thread_start_us":11931,"threads_started":1}
I20260926 07:44:21.408030 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.390056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.11.9.65:46863)
0926 07:44:21.402185 (+ 12129us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.402193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.402219 (+ 26us) server_negotiation.cc:407] Connection header received
0926 07:44:21.402312 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.402319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.402552 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.403073 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:21.403239 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.404187 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.406012 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.406996 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.407145 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.407371 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.407544 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.407693 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11947,"thread_start_us":11795,"threads_started":1}
I20260926 07:44:21.411265 30763 tablet_service.cc:1467] Processing CreateTablet for tablet b75f832453784e4d9e1e9f0affdb732e (DEFAULT_TABLE table=test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:21.412384 30763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75f832453784e4d9e1e9f0affdb732e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:21.421694 31114 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Bootstrap starting.
I20260926 07:44:21.422528 31113 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Bootstrap starting.
I20260926 07:44:21.424260 31115 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Bootstrap starting.
I20260926 07:44:21.426831 31113 tablet_bootstrap.cc:654] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:21.427527 31114 tablet_bootstrap.cc:654] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:21.427526 31115 tablet_bootstrap.cc:654] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:21.427865 31113 log.cc:824] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:21.428709 31115 log.cc:824] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:21.428710 31114 log.cc:824] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:21.430955 31113 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: No bootstrap required, opened a new log
I20260926 07:44:21.431243 31113 ts_tablet_manager.cc:1397] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 07:44:21.431521 31115 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: No bootstrap required, opened a new log
I20260926 07:44:21.431797 31115 ts_tablet_manager.cc:1397] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 07:44:21.432370 31114 tablet_bootstrap.cc:492] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: No bootstrap required, opened a new log
I20260926 07:44:21.432611 31114 ts_tablet_manager.cc:1397] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260926 07:44:21.439424 31113 raft_consensus.cc:348] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.439828 31113 raft_consensus.cc:374] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:21.439971 31113 raft_consensus.cc:729] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4e53efb1b743b089beb4d48cebcb3d, State: Initialized, Role: FOLLOWER
I20260926 07:44:21.440636 31113 consensus_queue.cc:260] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.442209 31095 heartbeater.cc:503] Master 127.11.9.126:45061 was elected leader, sending a full tablet report...
I20260926 07:44:21.442750 31113 ts_tablet_manager.cc:1428] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260926 07:44:21.442960 31115 raft_consensus.cc:348] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.443372 31115 raft_consensus.cc:374] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:21.443570 31115 raft_consensus.cc:729] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45cf78713364102b8a1c431a322dd76, State: Initialized, Role: FOLLOWER
I20260926 07:44:21.444164 31115 consensus_queue.cc:260] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.444324 31114 raft_consensus.cc:348] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.444890 31114 raft_consensus.cc:374] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:21.445935 31115 ts_tablet_manager.cc:1428] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260926 07:44:21.445039 31114 raft_consensus.cc:729] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e23973e4ae42ee880d75ab173ebc8f, State: Initialized, Role: FOLLOWER
I20260926 07:44:21.446817 31114 consensus_queue.cc:260] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.447921 30962 heartbeater.cc:503] Master 127.11.9.126:45061 was elected leader, sending a full tablet report...
I20260926 07:44:21.448818 31114 ts_tablet_manager.cc:1428] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260926 07:44:21.560106 30963 tablet.cc:2369] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:21.568452 31096 tablet.cc:2369] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:21.568420 30829 tablet.cc:2369] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:21.592538 31121 raft_consensus.cc:482] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:21.592818 31121 raft_consensus.cc:504] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.594476 31121 leader_election.cc:288] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f4e53efb1b743b089beb4d48cebcb3d (127.11.9.67:35347), f45cf78713364102b8a1c431a322dd76 (127.11.9.65:46863)
I20260926 07:44:21.601581 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.595770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:57567 (local address 127.11.9.65:46863)
0926 07:44:21.596016 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.596022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.596345 (+ 323us) server_negotiation.cc:407] Connection header received
0926 07:44:21.596538 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.596551 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.596610 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.596704 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:21.597242 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.597970 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.599772 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.600527 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.600682 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.600801 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.601193 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.601327 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 07:44:21.601624 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.594943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:53707 (local address 127.11.9.67:35347)
0926 07:44:21.595191 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.595199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.595577 (+ 378us) server_negotiation.cc:407] Connection header received
0926 07:44:21.595852 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.595859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.595952 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.596046 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:21.596706 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.597594 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.599708 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.600547 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.600704 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.600836 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.601194 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.601332 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 07:44:21.601758 31123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.595630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:46863 (local address 127.11.9.66:57567)
0926 07:44:21.596236 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.596249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.596396 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.596720 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.596724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.596746 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:21.597046 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.597053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.598102 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.598107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.599621 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.599633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.600780 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.600788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.601019 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.601042 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.601144 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.601320 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":70,"thread_start_us":287,"threads_started":1}
I20260926 07:44:21.601814 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.594871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:35347 (local address 127.11.9.66:53707)
0926 07:44:21.595467 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:21.595482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:21.595672 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:21.596129 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:21.596134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:21.596160 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:21.596537 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.596546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.597734 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.597743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.599520 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:21.599542 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.600794 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.600800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:21.601004 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.601029 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 07:44:21.601144 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 07:44:21.601332 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":14,"thread_start_us":309,"threads_started":1}
I20260926 07:44:21.602958 31050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75f832453784e4d9e1e9f0affdb732e" candidate_uuid: "60e23973e4ae42ee880d75ab173ebc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" is_pre_election: true
I20260926 07:44:21.602958 30783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75f832453784e4d9e1e9f0affdb732e" candidate_uuid: "60e23973e4ae42ee880d75ab173ebc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f45cf78713364102b8a1c431a322dd76" is_pre_election: true
I20260926 07:44:21.603518 31050 raft_consensus.cc:2454] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60e23973e4ae42ee880d75ab173ebc8f in term 0.
I20260926 07:44:21.603518 30783 raft_consensus.cc:2454] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60e23973e4ae42ee880d75ab173ebc8f in term 0.
I20260926 07:44:21.604169 30851 leader_election.cc:302] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e53efb1b743b089beb4d48cebcb3d, 60e23973e4ae42ee880d75ab173ebc8f; no voters:
I20260926 07:44:21.604509 31121 raft_consensus.cc:2781] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:21.604652 31121 raft_consensus.cc:482] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:21.604768 31121 raft_consensus.cc:3037] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:21.606950 31121 raft_consensus.cc:504] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.607513 31121 leader_election.cc:288] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [CANDIDATE]: Term 1 election: Requested vote from peers 5f4e53efb1b743b089beb4d48cebcb3d (127.11.9.67:35347), f45cf78713364102b8a1c431a322dd76 (127.11.9.65:46863)
I20260926 07:44:21.607892 31050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75f832453784e4d9e1e9f0affdb732e" candidate_uuid: "60e23973e4ae42ee880d75ab173ebc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f4e53efb1b743b089beb4d48cebcb3d"
I20260926 07:44:21.607973 30783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75f832453784e4d9e1e9f0affdb732e" candidate_uuid: "60e23973e4ae42ee880d75ab173ebc8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f45cf78713364102b8a1c431a322dd76"
I20260926 07:44:21.608119 31050 raft_consensus.cc:3037] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:21.608202 30783 raft_consensus.cc:3037] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:21.610419 30783 raft_consensus.cc:2454] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60e23973e4ae42ee880d75ab173ebc8f in term 1.
I20260926 07:44:21.610661 31050 raft_consensus.cc:2454] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60e23973e4ae42ee880d75ab173ebc8f in term 1.
I20260926 07:44:21.610966 30851 leader_election.cc:302] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60e23973e4ae42ee880d75ab173ebc8f, f45cf78713364102b8a1c431a322dd76; no voters:
I20260926 07:44:21.611289 31121 raft_consensus.cc:2781] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:21.611768 31121 raft_consensus.cc:686] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 1 LEADER]: Becoming Leader. State: Replica: 60e23973e4ae42ee880d75ab173ebc8f, State: Running, Role: LEADER
I20260926 07:44:21.612371 31121 consensus_queue.cc:237] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } }
I20260926 07:44:21.617066 30639 catalog_manager.cc:5495] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 60e23973e4ae42ee880d75ab173ebc8f (127.11.9.66). New cstate: current_term: 1 leader_uuid: "60e23973e4ae42ee880d75ab173ebc8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e23973e4ae42ee880d75ab173ebc8f" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 35009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 } health_report { overall_health: UNKNOWN } } }
I20260926 07:44:21.634513 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.629108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.11.9.66:35009)
0926 07:44:21.629380 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.629385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.629403 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:44:21.629494 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.629498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.629577 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.629704 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:21.630035 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.630818 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.632146 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.632574 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.632689 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.633344 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:21.633863 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:21.633986 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.634116 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.634180 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 07:44:21.642680 30783 raft_consensus.cc:1261] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 1 FOLLOWER]: Refusing update from remote peer 60e23973e4ae42ee880d75ab173ebc8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:21.642807 31050 raft_consensus.cc:1261] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 1 FOLLOWER]: Refusing update from remote peer 60e23973e4ae42ee880d75ab173ebc8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:21.643731 31124 consensus_queue.cc:1035] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45cf78713364102b8a1c431a322dd76" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:21.644127 31121 consensus_queue.cc:1035] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f4e53efb1b743b089beb4d48cebcb3d" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 35347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:21.657881 31127 mvcc.cc:205] Tried to move back new op lower bound from 7333513878078742528 to 7333513877968781312. Current Snapshot: MvccSnapshot[applied={T|T < 7333513878078742528}]
I20260926 07:44:21.663784 31129 mvcc.cc:205] Tried to move back new op lower bound from 7333513878078742528 to 7333513877968781312. Current Snapshot: MvccSnapshot[applied={T|T < 7333513878078742528}]
I20260926 07:44:21.686836 30638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:21.687182 30638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:21.691169 30638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac2d38a92b454594878e4cc61a31ec81: Sending DeleteTablet for 3 replicas of tablet b75f832453784e4d9e1e9f0affdb732e
I20260926 07:44:21.692430 30897 tablet_service.cc:1514] Processing DeleteTablet for tablet b75f832453784e4d9e1e9f0affdb732e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:21 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:59550
I20260926 07:44:21.692639 30763 tablet_service.cc:1514] Processing DeleteTablet for tablet b75f832453784e4d9e1e9f0affdb732e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:21 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:49270
I20260926 07:44:21.692792 31030 tablet_service.cc:1514] Processing DeleteTablet for tablet b75f832453784e4d9e1e9f0affdb732e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:21 UTC) from {username='kudu', principal='kudu/127.11.9.126@KRBTEST.COM'} at 127.0.0.1:60192
I20260926 07:44:21.694531 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30700
I20260926 07:44:21.703697 31138 tablet_replica.cc:330] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: stopping tablet replica
I20260926 07:44:21.704339 31138 raft_consensus.cc:2229] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:21.705039 31138 raft_consensus.cc:2258] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:21.707554 31138 ts_tablet_manager.cc:1905] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:21.716214 31138 ts_tablet_manager.cc:1918] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:21.723681 31138 log.cc:1196] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-0/wal/wals/b75f832453784e4d9e1e9f0affdb732e
I20260926 07:44:21.724548 31138 ts_tablet_manager.cc:1939] T b75f832453784e4d9e1e9f0affdb732e P f45cf78713364102b8a1c431a322dd76: Deleting consensus metadata
I20260926 07:44:21.727236 31137 tablet_replica.cc:330] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: stopping tablet replica
I20260926 07:44:21.727396 30624 catalog_manager.cc:4841] TS f45cf78713364102b8a1c431a322dd76 (127.11.9.65:46863): tablet b75f832453784e4d9e1e9f0affdb732e (table test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]) successfully deleted
I20260926 07:44:21.727952 31139 tablet_replica.cc:330] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: stopping tablet replica
I20260926 07:44:21.728435 31139 raft_consensus.cc:2229] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:21.729065 31137 raft_consensus.cc:2229] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:44:21.729815 31137 raft_consensus.cc:2258] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:21.730729 31139 raft_consensus.cc:2258] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:21.731842 31137 ts_tablet_manager.cc:1905] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:21.733634 31139 ts_tablet_manager.cc:1905] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:21.739183 31137 ts_tablet_manager.cc:1918] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:21.739426 31137 log.cc:1196] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/b75f832453784e4d9e1e9f0affdb732e
Sep 26 07:44:21 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.65@KRBTEST.COM
I20260926 07:44:21.740002 31137 ts_tablet_manager.cc:1939] T b75f832453784e4d9e1e9f0affdb732e P 60e23973e4ae42ee880d75ab173ebc8f: Deleting consensus metadata
I20260926 07:44:21.740438 31139 ts_tablet_manager.cc:1918] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:21.740658 31139 log.cc:1196] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/b75f832453784e4d9e1e9f0affdb732e
I20260926 07:44:21.741345 31139 ts_tablet_manager.cc:1939] T b75f832453784e4d9e1e9f0affdb732e P 5f4e53efb1b743b089beb4d48cebcb3d: Deleting consensus metadata
I20260926 07:44:21.741842 30623 catalog_manager.cc:4841] TS 60e23973e4ae42ee880d75ab173ebc8f (127.11.9.66:35009): tablet b75f832453784e4d9e1e9f0affdb732e (table test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]) successfully deleted
I20260926 07:44:21.743395 30624 catalog_manager.cc:4841] TS 5f4e53efb1b743b089beb4d48cebcb3d (127.11.9.67:35347): tablet b75f832453784e4d9e1e9f0affdb732e (table test-table [id=c2d2d560e0cd41bb938b76cada65ab0c]) successfully deleted
I20260926 07:44:21.749357 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.696062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.11.9.65:46863)
0926 07:44:21.696290 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.696298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.719244 (+ 22946us) server_negotiation.cc:407] Connection header received
0926 07:44:21.719842 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.719851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.719958 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.720055 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:21.722403 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.723787 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.728708 (+ 4921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.729365 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.729497 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.741596 (+ 12099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:21.741615 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:21.741634 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:21.741682 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:21.745934 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.746977 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.746986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.746989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.747140 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.747577 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.747582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.747585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.748078 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:21.748380 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.748715 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.748963 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:44:21.750103 30823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:21.839746 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30700
I20260926 07:44:21.858106 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30834
Sep 26 07:44:21 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.66@KRBTEST.COM
I20260926 07:44:21.875093 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:21.858889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.11.9.66:35009)
0926 07:44:21.859090 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:21.859096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:21.859144 (+ 48us) server_negotiation.cc:407] Connection header received
0926 07:44:21.859381 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:21.859387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:21.859463 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:21.859568 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:21.860802 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.861736 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.863011 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:21.863475 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:21.863609 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:21.868949 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:21.868961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:21.868975 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:21.869015 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:21.872151 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.872858 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.872865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.872867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.873054 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:21.873511 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:21.873520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:21.873524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:21.873893 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:21.874241 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:21.874508 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 07:44:21.874740 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 07:44:21.875733 30957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:21.994146 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30834
I20260926 07:44:22.012981 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30967
Sep 26 07:44:22 dist-test-slave-x492 krb5kdc[30594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.67@KRBTEST.COM
I20260926 07:44:22.027872 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:22.013753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.11.9.67:35347)
0926 07:44:22.013949 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:22.013954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:22.013972 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:44:22.014080 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:22.014083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:22.014148 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:22.014238 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:22.015484 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:22.016199 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:22.017516 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:22.017887 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:22.017982 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:22.022471 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:22.022485 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:22.022500 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:22.022534 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:22.025447 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:22.026095 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:22.026112 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:22.026116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:22.026293 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:22.026651 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:22.026655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:22.026658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:22.027017 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:22.027271 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:22.027457 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 07:44:22.027616 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:44:22.028555 31090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:22.118070 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30967
I20260926 07:44:22.134887 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 30607
I20260926 07:44:22.135987 30669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:22.246374 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 30607
2026-09-26T07:44:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4313 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T07:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:22Z Disabled control of system clock
I20260926 07:44:22.320151 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:43571
--webserver_interface=127.11.9.126
--webserver_port=0
--builtin_ntp_servers=127.11.9.84:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:43571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:22.646538 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:22.646939 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:22.647078 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:22.654218 31148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:22.654313 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:22.654350 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:22.654409 31148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:22.654452 31148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:22.663439 31148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:43571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:43571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:22.664673 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:22.666491 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:22.677937 31153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:22.678541 31156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:22.678562 31154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:22.679754 31148 server_base.cc:1034] running on GCE node
I20260926 07:44:22.680454 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:22.681849 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:22.683063 31148 hybrid_clock.cc:648] HybridClock initialized: now 1790408662683012 us; error 44 us; skew 500 ppm
I20260926 07:44:22.683439 31148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:22.686008 31148 webserver.cc:460] Webserver started at http://127.11.9.126:36021/ using document root <none> and password file <none>
I20260926 07:44:22.686708 31148 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:22.686796 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:22.687162 31148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:22.690369 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "467a7fd786814d3eb110715bc9b50ae8"
format_stamp: "Formatted at 2026-09-26 07:44:22 on dist-test-slave-x492"
server_key: "e25515933ce59f9298d06ad5aa788aa9"
server_key_iv: "d8eb0fee71503be4b13cdbe7ea85fb7f"
server_key_version: "encryptionkey@0"
I20260926 07:44:22.691215 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "467a7fd786814d3eb110715bc9b50ae8"
format_stamp: "Formatted at 2026-09-26 07:44:22 on dist-test-slave-x492"
server_key: "e25515933ce59f9298d06ad5aa788aa9"
server_key_iv: "d8eb0fee71503be4b13cdbe7ea85fb7f"
server_key_version: "encryptionkey@0"
I20260926 07:44:22.696131 31148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:44:22.699571 31162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:22.700977 31148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 07:44:22.701161 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "467a7fd786814d3eb110715bc9b50ae8"
format_stamp: "Formatted at 2026-09-26 07:44:22 on dist-test-slave-x492"
server_key: "e25515933ce59f9298d06ad5aa788aa9"
server_key_iv: "d8eb0fee71503be4b13cdbe7ea85fb7f"
server_key_version: "encryptionkey@0"
I20260926 07:44:22.701434 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:22.713935 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:22.714743 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:22.715034 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:22.735313 31148 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:43571
I20260926 07:44:22.735365 31213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:43571 every 8 connection(s)
I20260926 07:44:22.736951 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:22.740005 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31148
I20260926 07:44:22.740206 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:22.740712 11301 external_mini_cluster.cc:1428] Setting key c87f3fb916cfb5b8b2fa40ff8052a083
I20260926 07:44:22.740839 31214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:22.753494 31214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: Bootstrap starting.
I20260926 07:44:22.753947 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:22.742560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.11.9.126:43571)
0926 07:44:22.743533 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:22.743551 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:22.743595 (+ 44us) server_negotiation.cc:407] Connection header received
0926 07:44:22.745480 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:22.745501 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:22.745952 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:22.746341 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:22.746912 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:22.748217 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:22.749621 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:22.750288 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:22.750515 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:22.751668 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:22.751687 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:22.751699 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:22.752207 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:22.752280 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:22.752304 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:22.752429 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:22.752628 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 07:44:22.752916 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":360,"threads_started":1}
I20260926 07:44:22.756935 31214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:22.757939 31214 log.cc:824] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:22.760474 31214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: No bootstrap required, opened a new log
I20260926 07:44:22.768004 31214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } }
I20260926 07:44:22.768360 31214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:22.768476 31214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467a7fd786814d3eb110715bc9b50ae8, State: Initialized, Role: FOLLOWER
I20260926 07:44:22.769119 31214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } }
I20260926 07:44:22.769412 31214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:22.769559 31214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:22.769747 31214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:22.771780 31214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } }
I20260926 07:44:22.772279 31214 leader_election.cc:302] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 467a7fd786814d3eb110715bc9b50ae8; no voters:
I20260926 07:44:22.772881 31214 leader_election.cc:288] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:22.773012 31219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:22.773845 31219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [term 1 LEADER]: Becoming Leader. State: Replica: 467a7fd786814d3eb110715bc9b50ae8, State: Running, Role: LEADER
I20260926 07:44:22.774374 31219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } }
I20260926 07:44:22.775125 31214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:22.777132 31221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467a7fd786814d3eb110715bc9b50ae8. Latest consensus state: current_term: 1 leader_uuid: "467a7fd786814d3eb110715bc9b50ae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } } }
I20260926 07:44:22.777639 31221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:22.777477 31220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "467a7fd786814d3eb110715bc9b50ae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467a7fd786814d3eb110715bc9b50ae8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 43571 } } }
I20260926 07:44:22.777855 31220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:22.779660 31227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:22.784904 31227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:22.793323 31227 catalog_manager.cc:1347] Generated new cluster ID: 57e6311fbe9c4a1396f60dc4c87e331a
I20260926 07:44:22.793481 31227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:22.822324 31227 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:22.823738 31227 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:22.832240 31227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: Generated new TSK 0
I20260926 07:44:22.833055 31227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:22.838987 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.65:0
--local_ip_for_outbound_sockets=127.11.9.65
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--builtin_ntp_servers=127.11.9.84:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 07:44:23.181383 31238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:23.181653 31238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:23.181737 31238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:23.188846 31238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:23.188946 31238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:23.189052 31238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.65
I20260926 07:44:23.198290 31238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.9.65
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.65
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:23.199508 31238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:23.201210 31238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:23.214051 31243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:23.214003 31246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:23.214109 31244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:23.215140 31238 server_base.cc:1034] running on GCE node
I20260926 07:44:23.215731 31238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:23.216693 31238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:23.217886 31238 hybrid_clock.cc:648] HybridClock initialized: now 1790408663217825 us; error 55 us; skew 500 ppm
I20260926 07:44:23.218242 31238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:23.220347 31238 webserver.cc:460] Webserver started at http://127.11.9.65:38833/ using document root <none> and password file <none>
I20260926 07:44:23.220964 31238 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:23.221047 31238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:23.221374 31238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:23.224082 31238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data/instance:
uuid: "575c57a5e9e34c72b8d261ef1abc6e7e"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "f18611827d241ac46b50605edb33455c"
server_key_iv: "46b1d4ec2ffcba3ef394531e19a84cd9"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.224788 31238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance:
uuid: "575c57a5e9e34c72b8d261ef1abc6e7e"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "f18611827d241ac46b50605edb33455c"
server_key_iv: "46b1d4ec2ffcba3ef394531e19a84cd9"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.229684 31238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 07:44:23.232893 31252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.234258 31238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:23.234443 31238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal
uuid: "575c57a5e9e34c72b8d261ef1abc6e7e"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "f18611827d241ac46b50605edb33455c"
server_key_iv: "46b1d4ec2ffcba3ef394531e19a84cd9"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.234651 31238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:23.260372 31238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:23.261252 31238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:23.261538 31238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:23.262467 31238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:23.263895 31238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:23.264014 31238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.264148 31238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:23.264240 31238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.296927 31238 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.65:37791
I20260926 07:44:23.297039 31364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.65:37791 every 8 connection(s)
I20260926 07:44:23.298305 31238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/data/info.pb
I20260926 07:44:23.306738 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31238
I20260926 07:44:23.306917 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-0/wal/instance
I20260926 07:44:23.307437 11301 external_mini_cluster.cc:1428] Setting key dbac3ba8570e30ee417a4a74f1196f76
I20260926 07:44:23.310623 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.66:0
--local_ip_for_outbound_sockets=127.11.9.66
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--builtin_ntp_servers=127.11.9.84:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:23.311925 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:23.300731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:33757 (local address 127.11.9.126:43571)
0926 07:44:23.301136 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:23.301140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:23.302478 (+ 1338us) server_negotiation.cc:407] Connection header received
0926 07:44:23.304336 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:23.304341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:23.304415 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:23.304557 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:23.305661 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.306404 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.308590 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.309261 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.309397 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:23.310963 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:23.310971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:23.310978 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:23.311001 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:23.311048 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:23.311065 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:23.311155 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:23.311381 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 07:44:23.311490 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":199,"threads_started":1}
I20260926 07:44:23.312708 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:23.301424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43571 (local address 127.11.9.65:33757)
0926 07:44:23.302302 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:23.302363 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:23.304082 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:23.304753 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:23.304761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:23.304858 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:23.305486 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:23.305503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.306557 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.306562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:23.308450 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:23.308470 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.309492 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.309499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:23.309779 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:23.310565 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:23.310570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:23.310622 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:23.310732 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:23.311170 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:23.311179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:23.311186 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 07:44:23.311336 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 07:44:23.311612 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":289,"threads_started":1}
I20260926 07:44:23.314147 31365 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43571
I20260926 07:44:23.314462 31365 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:23.315343 31365 heartbeater.cc:511] Master 127.11.9.126:43571 requested a full tablet report, sending...
I20260926 07:44:23.317552 31179 ts_manager.cc:194] Registered new tserver with Master: 575c57a5e9e34c72b8d261ef1abc6e7e (127.11.9.65:37791)
I20260926 07:44:23.319026 31179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.65:33757
W20260926 07:44:23.633240 31369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:23.633499 31369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:23.633641 31369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:23.640950 31369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:23.641041 31369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:23.641279 31369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.66
I20260926 07:44:23.650108 31369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.9.66
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.66
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:23.651316 31369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:23.653102 31369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:23.665962 31374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:23.666041 31377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:23.666000 31375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:23.666877 31369 server_base.cc:1034] running on GCE node
I20260926 07:44:23.667414 31369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:23.668325 31369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:23.669519 31369 hybrid_clock.cc:648] HybridClock initialized: now 1790408663669478 us; error 46 us; skew 500 ppm
I20260926 07:44:23.669871 31369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:23.672060 31369 webserver.cc:460] Webserver started at http://127.11.9.66:34705/ using document root <none> and password file <none>
I20260926 07:44:23.672755 31369 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:23.672868 31369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:23.673237 31369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:23.675969 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data/instance:
uuid: "3e81144ce19847f688b90265c659cd03"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "8e3d8d9f542abfbb7090d6139ff53c24"
server_key_iv: "efe484a6c3bae8d9d7c10450c0b78284"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.676748 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance:
uuid: "3e81144ce19847f688b90265c659cd03"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "8e3d8d9f542abfbb7090d6139ff53c24"
server_key_iv: "efe484a6c3bae8d9d7c10450c0b78284"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.681876 31369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 07:44:23.685348 31383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.686769 31369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 07:44:23.686949 31369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal
uuid: "3e81144ce19847f688b90265c659cd03"
format_stamp: "Formatted at 2026-09-26 07:44:23 on dist-test-slave-x492"
server_key: "8e3d8d9f542abfbb7090d6139ff53c24"
server_key_iv: "efe484a6c3bae8d9d7c10450c0b78284"
server_key_version: "encryptionkey@0"
I20260926 07:44:23.687155 31369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:23.704546 31369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:23.705567 31369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:23.705849 31369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:23.706914 31369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:23.708400 31369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:23.708524 31369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.708658 31369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:23.708760 31369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:23.746423 31369 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.66:34917
I20260926 07:44:23.746467 31495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.66:34917 every 8 connection(s)
I20260926 07:44:23.747722 31369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/data/info.pb
I20260926 07:44:23.755252 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31369
I20260926 07:44:23.755439 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal/instance
I20260926 07:44:23.755885 11301 external_mini_cluster.cc:1428] Setting key a417a7b57e0095915abafc39b5df160e
I20260926 07:44:23.758526 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.9.67:0
--local_ip_for_outbound_sockets=127.11.9.67
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--builtin_ntp_servers=127.11.9.84:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 07:44:23.760627 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:23.750530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.66:53881 (local address 127.11.9.126:43571)
0926 07:44:23.750734 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:23.750738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:23.752228 (+ 1490us) server_negotiation.cc:407] Connection header received
0926 07:44:23.754055 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:23.754060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:23.754151 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:23.754256 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:23.755305 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.756125 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.757944 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.758319 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.758452 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:23.759847 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:23.759852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:23.759855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:23.759871 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:23.759901 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:23.759912 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:23.760008 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:23.760245 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 07:44:23.760332 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:44:23.761646 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:23.751106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43571 (local address 127.11.9.66:53881)
0926 07:44:23.752053 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:23.752093 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:23.753798 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:23.754488 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:23.754496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:23.754580 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:23.755137 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:23.755152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.756263 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.756269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:23.757799 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:23.757810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:23.758530 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:23.758537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:23.758854 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:23.759552 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:23.759554 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:23.759589 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:23.759658 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:23.760020 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:23.760032 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:23.760040 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 07:44:23.760223 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 07:44:23.760583 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":299,"threads_started":1}
I20260926 07:44:23.763001 31496 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43571
I20260926 07:44:23.763309 31496 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:23.764184 31496 heartbeater.cc:511] Master 127.11.9.126:43571 requested a full tablet report, sending...
I20260926 07:44:23.765746 31179 ts_manager.cc:194] Registered new tserver with Master: 3e81144ce19847f688b90265c659cd03 (127.11.9.66:34917)
I20260926 07:44:23.766634 31179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.66:53881
W20260926 07:44:24.090979 31499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:24.091306 31499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:24.091480 31499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:24.098694 31499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:24.098832 31499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:24.099009 31499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.9.67
I20260926 07:44:24.108358 31499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.9.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.9.67
--webserver_port=0
--tserver_master_addrs=127.11.9.126:43571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.9.67
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:24.109691 31499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:24.111413 31499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:24.124547 31504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:24.124521 31505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:24.124802 31507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:24.125322 31499 server_base.cc:1034] running on GCE node
I20260926 07:44:24.125842 31499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:24.126703 31499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:24.127905 31499 hybrid_clock.cc:648] HybridClock initialized: now 1790408664127853 us; error 54 us; skew 500 ppm
I20260926 07:44:24.128208 31499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:44:24.130482 31499 webserver.cc:460] Webserver started at http://127.11.9.67:44505/ using document root <none> and password file <none>
I20260926 07:44:24.131112 31499 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:24.131204 31499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:24.131454 31499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:24.134372 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data/instance:
uuid: "f70a4d395f7f4005beeeedae6b134fc3"
format_stamp: "Formatted at 2026-09-26 07:44:24 on dist-test-slave-x492"
server_key: "d7f834e62d3da2b89c31d04ba478c900"
server_key_iv: "2599efaa102fac685caac09f1689ec23"
server_key_version: "encryptionkey@0"
I20260926 07:44:24.135138 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance:
uuid: "f70a4d395f7f4005beeeedae6b134fc3"
format_stamp: "Formatted at 2026-09-26 07:44:24 on dist-test-slave-x492"
server_key: "d7f834e62d3da2b89c31d04ba478c900"
server_key_iv: "2599efaa102fac685caac09f1689ec23"
server_key_version: "encryptionkey@0"
I20260926 07:44:24.140393 31499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:44:24.143913 31513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:24.145360 31499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:24.145541 31499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal
uuid: "f70a4d395f7f4005beeeedae6b134fc3"
format_stamp: "Formatted at 2026-09-26 07:44:24 on dist-test-slave-x492"
server_key: "d7f834e62d3da2b89c31d04ba478c900"
server_key_iv: "2599efaa102fac685caac09f1689ec23"
server_key_version: "encryptionkey@0"
I20260926 07:44:24.145788 31499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:24.157903 31499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:24.158656 31499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:24.158979 31499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:24.159888 31499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:44:24.161370 31499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:44:24.161484 31499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:24.161613 31499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:44:24.161700 31499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:24.195407 31499 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.67:42227
I20260926 07:44:24.195457 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.67:42227 every 8 connection(s)
I20260926 07:44:24.196730 31499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/data/info.pb
I20260926 07:44:24.204802 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31499
I20260926 07:44:24.204970 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal/instance
I20260926 07:44:24.205507 11301 external_mini_cluster.cc:1428] Setting key fdd21ecc07178892b61bfa618e52e32a
I20260926 07:44:24.209818 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.199376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.67:47199 (local address 127.11.9.126:43571)
0926 07:44:24.199544 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.199548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.200810 (+ 1262us) server_negotiation.cc:407] Connection header received
0926 07:44:24.202590 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.202594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.202663 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.202757 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:24.203732 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.204564 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.206679 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.207270 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.207395 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.209012 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:24.209017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:24.209020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:24.209043 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:24.209076 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:24.209088 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:24.209212 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.209451 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.209527 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 07:44:24.210745 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.199774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:43571 (local address 127.11.9.67:47199)
0926 07:44:24.200669 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.200710 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.202355 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.202949 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.202957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.203037 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:44:24.203586 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.203602 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.204771 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.204776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.206536 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.206551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.207531 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.207538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.207898 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.208721 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 07:44:24.208726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:44:24.208765 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 07:44:24.208838 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:44:24.209254 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:44:24.209271 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:44:24.209277 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.209425 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.209706 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":295,"threads_started":1}
I20260926 07:44:24.212014 31626 heartbeater.cc:347] Connected to a master server at 127.11.9.126:43571
I20260926 07:44:24.212306 31626 heartbeater.cc:464] Registering TS with master...
I20260926 07:44:24.213011 31626 heartbeater.cc:511] Master 127.11.9.126:43571 requested a full tablet report, sending...
I20260926 07:44:24.214427 31179 ts_manager.cc:194] Registered new tserver with Master: f70a4d395f7f4005beeeedae6b134fc3 (127.11.9.67:42227)
I20260926 07:44:24.215240 31179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.9.67:47199
I20260926 07:44:24.220695 11301 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:44:24.230638 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.224762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.11.9.126:43571)
0926 07:44:24.225025 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.225032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.225363 (+ 331us) server_negotiation.cc:407] Connection header received
0926 07:44:24.225507 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.225511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.225581 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.225661 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:24.226217 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.226958 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.228270 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.228688 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.228781 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.229967 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:24.229972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:24.229975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:24.229991 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:24.230026 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:24.230038 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:24.230120 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.230242 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.230311 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:44:24.234174 31179 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:44:24.240971 31179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:44:24.270854 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.253047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.11.9.66:34917)
0926 07:44:24.254122 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.254134 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.254191 (+ 57us) server_negotiation.cc:407] Connection header received
0926 07:44:24.254303 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.254312 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.254596 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.254828 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:24.257423 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.260972 (+ 3549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.267026 (+ 6054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.268510 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.268649 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.268919 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.270316 (+ 1397us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.270499 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":765,"threads_started":1}
I20260926 07:44:24.271145 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.252814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:34917 (local address 127.0.0.1:56524)
0926 07:44:24.253865 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.253904 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.254067 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.255187 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.255199 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.255246 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:24.257143 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.257190 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.261226 (+ 4036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.261236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.266839 (+ 5603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.266854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.268740 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.268748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.269007 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.270085 (+ 1078us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.270213 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.270548 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":110,"thread_start_us":534,"threads_started":1}
I20260926 07:44:24.274844 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.254263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:42227 (local address 127.0.0.1:49648)
0926 07:44:24.254840 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.254862 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.255001 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.267910 (+ 12909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.267916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.267943 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:24.268302 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.268315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.269885 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.269892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.272176 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.272190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.273959 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.273966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.274154 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.274212 (+ 58us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.274316 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.274451 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":65,"thread_start_us":239,"threads_started":1}
I20260926 07:44:24.274904 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.254512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.11.9.67:42227)
0926 07:44:24.267283 (+ 12771us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.267290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.267320 (+ 30us) server_negotiation.cc:407] Connection header received
0926 07:44:24.267401 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.267407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.267694 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.267915 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:24.268541 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.269755 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.272396 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.273753 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.273928 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.274222 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.274358 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.274526 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12630,"thread_start_us":12478,"threads_started":1}
I20260926 07:44:24.275480 31431 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6fe594977840478df1544885366729 (DEFAULT_TABLE table=test-table [id=63154adf0dfd449f8e12d1a6c3554c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:24.276782 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.255426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.11.9.65:37791)
0926 07:44:24.267047 (+ 11621us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.267069 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.267581 (+ 512us) server_negotiation.cc:407] Connection header received
0926 07:44:24.267752 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.267760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.268022 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.268243 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:24.268950 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.270581 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.273317 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.274691 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.275912 (+ 1221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.276183 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.276264 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.276440 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11414,"thread_start_us":11266,"threads_started":1}
I20260926 07:44:24.276991 31431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6fe594977840478df1544885366729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:24.277036 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.254858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:55460)
0926 07:44:24.267436 (+ 12578us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.267455 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.267580 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.268264 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.268269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.268298 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:24.268749 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.268760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.270762 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.270768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.273016 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.273031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.275348 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.275354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.275555 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.275594 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.275674 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.275785 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12464,"mutex_wait_us":1,"thread_start_us":11803,"threads_started":1}
I20260926 07:44:24.279358 31561 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6fe594977840478df1544885366729 (DEFAULT_TABLE table=test-table [id=63154adf0dfd449f8e12d1a6c3554c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:24.280684 31561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6fe594977840478df1544885366729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:24.280576 31300 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6fe594977840478df1544885366729 (DEFAULT_TABLE table=test-table [id=63154adf0dfd449f8e12d1a6c3554c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:44:24.281849 31300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6fe594977840478df1544885366729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:24.292917 31644 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Bootstrap starting.
I20260926 07:44:24.294545 31645 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Bootstrap starting.
I20260926 07:44:24.297190 31644 tablet_bootstrap.cc:654] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:24.297467 31646 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Bootstrap starting.
I20260926 07:44:24.298256 31645 tablet_bootstrap.cc:654] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:24.298652 31644 log.cc:824] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:24.299541 31645 log.cc:824] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:24.301504 31646 tablet_bootstrap.cc:654] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:24.303094 31646 log.cc:824] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:24.309335 31644 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: No bootstrap required, opened a new log
I20260926 07:44:24.309576 31645 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: No bootstrap required, opened a new log
I20260926 07:44:24.309679 31644 ts_tablet_manager.cc:1397] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260926 07:44:24.309978 31645 ts_tablet_manager.cc:1397] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260926 07:44:24.318923 31644 raft_consensus.cc:348] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.319388 31644 raft_consensus.cc:374] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:24.319504 31644 raft_consensus.cc:729] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e81144ce19847f688b90265c659cd03, State: Initialized, Role: FOLLOWER
I20260926 07:44:24.319937 31646 tablet_bootstrap.cc:492] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: No bootstrap required, opened a new log
I20260926 07:44:24.320202 31646 ts_tablet_manager.cc:1397] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.017s
I20260926 07:44:24.320125 31644 consensus_queue.cc:260] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.321298 31496 heartbeater.cc:503] Master 127.11.9.126:43571 was elected leader, sending a full tablet report...
I20260926 07:44:24.321977 31644 ts_tablet_manager.cc:1428] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260926 07:44:24.321920 31645 raft_consensus.cc:348] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.322301 31645 raft_consensus.cc:374] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:24.322419 31645 raft_consensus.cc:729] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70a4d395f7f4005beeeedae6b134fc3, State: Initialized, Role: FOLLOWER
I20260926 07:44:24.322448 31365 heartbeater.cc:503] Master 127.11.9.126:43571 was elected leader, sending a full tablet report...
I20260926 07:44:24.323173 31645 consensus_queue.cc:260] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.325707 31645 ts_tablet_manager.cc:1428] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260926 07:44:24.325742 31626 heartbeater.cc:503] Master 127.11.9.126:43571 was elected leader, sending a full tablet report...
I20260926 07:44:24.332491 31646 raft_consensus.cc:348] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.332875 31646 raft_consensus.cc:374] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:24.332983 31646 raft_consensus.cc:729] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575c57a5e9e34c72b8d261ef1abc6e7e, State: Initialized, Role: FOLLOWER
I20260926 07:44:24.333547 31646 consensus_queue.cc:260] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.334890 31646 ts_tablet_manager.cc:1428] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
W20260926 07:44:24.448508 31627 tablet.cc:2369] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:44:24.499784 31497 tablet.cc:2369] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:24.514487 31652 raft_consensus.cc:482] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:44:24.514858 31652 raft_consensus.cc:504] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.516521 31652 leader_election.cc:288] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e81144ce19847f688b90265c659cd03 (127.11.9.66:34917), f70a4d395f7f4005beeeedae6b134fc3 (127.11.9.67:42227)
I20260926 07:44:24.523448 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.517223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:36353 (local address 127.11.9.66:34917)
0926 07:44:24.517415 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.517419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.517945 (+ 526us) server_negotiation.cc:407] Connection header received
0926 07:44:24.518168 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.518175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.518270 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.518377 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:24.519106 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.519807 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.521628 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.522539 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.522676 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.522766 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.523052 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.523179 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:44:24.523520 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.517081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.66:34917 (local address 127.11.9.65:36353)
0926 07:44:24.517834 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.517856 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.518003 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.518521 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.518526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.518553 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:24.518962 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.518971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.519942 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.519950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.521451 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.521462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.522712 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.522719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.522894 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.522919 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.523022 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.523207 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":436,"threads_started":1}
I20260926 07:44:24.523794 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.518284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.9.65:35539 (local address 127.11.9.67:42227)
0926 07:44:24.518474 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.518479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.518694 (+ 215us) server_negotiation.cc:407] Connection header received
0926 07:44:24.518894 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.518905 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.518978 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.519069 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 07:44:24.519621 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.520304 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.522072 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.522889 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.523017 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.523117 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.523454 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.523572 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:44:24.523852 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.517969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.67:42227 (local address 127.11.9.65:35539)
0926 07:44:24.518586 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.518610 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 07:44:24.518754 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:44:24.519127 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:44:24.519131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:44:24.519159 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 07:44:24.519451 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.519459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.520442 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.520447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.521870 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:44:24.521882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.523078 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.523084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:44:24.523280 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.523326 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 07:44:24.523424 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 07:44:24.523546 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":319,"threads_started":1}
I20260926 07:44:24.524768 31581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6fe594977840478df1544885366729" candidate_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70a4d395f7f4005beeeedae6b134fc3" is_pre_election: true
I20260926 07:44:24.524830 31451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6fe594977840478df1544885366729" candidate_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e81144ce19847f688b90265c659cd03" is_pre_election: true
I20260926 07:44:24.525345 31581 raft_consensus.cc:2454] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575c57a5e9e34c72b8d261ef1abc6e7e in term 0.
I20260926 07:44:24.525352 31451 raft_consensus.cc:2454] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575c57a5e9e34c72b8d261ef1abc6e7e in term 0.
I20260926 07:44:24.525995 31254 leader_election.cc:302] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575c57a5e9e34c72b8d261ef1abc6e7e, f70a4d395f7f4005beeeedae6b134fc3; no voters:
I20260926 07:44:24.526324 31652 raft_consensus.cc:2781] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:44:24.526486 31652 raft_consensus.cc:482] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:44:24.526600 31652 raft_consensus.cc:3037] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:24.528698 31652 raft_consensus.cc:504] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.529364 31652 leader_election.cc:288] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [CANDIDATE]: Term 1 election: Requested vote from peers 3e81144ce19847f688b90265c659cd03 (127.11.9.66:34917), f70a4d395f7f4005beeeedae6b134fc3 (127.11.9.67:42227)
I20260926 07:44:24.529691 31451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6fe594977840478df1544885366729" candidate_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e81144ce19847f688b90265c659cd03"
I20260926 07:44:24.529891 31451 raft_consensus.cc:3037] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:24.529835 31581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6fe594977840478df1544885366729" candidate_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70a4d395f7f4005beeeedae6b134fc3"
I20260926 07:44:24.530061 31581 raft_consensus.cc:3037] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:24.532029 31451 raft_consensus.cc:2454] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575c57a5e9e34c72b8d261ef1abc6e7e in term 1.
I20260926 07:44:24.532330 31581 raft_consensus.cc:2454] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575c57a5e9e34c72b8d261ef1abc6e7e in term 1.
I20260926 07:44:24.532431 31254 leader_election.cc:302] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e81144ce19847f688b90265c659cd03, 575c57a5e9e34c72b8d261ef1abc6e7e; no voters:
I20260926 07:44:24.532703 31652 raft_consensus.cc:2781] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:24.533147 31652 raft_consensus.cc:686] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 1 LEADER]: Becoming Leader. State: Replica: 575c57a5e9e34c72b8d261ef1abc6e7e, State: Running, Role: LEADER
I20260926 07:44:24.533666 31652 consensus_queue.cc:237] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } }
I20260926 07:44:24.538378 31179 catalog_manager.cc:5495] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 575c57a5e9e34c72b8d261ef1abc6e7e (127.11.9.65). New cstate: current_term: 1 leader_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575c57a5e9e34c72b8d261ef1abc6e7e" member_type: VOTER last_known_addr { host: "127.11.9.65" port: 37791 } health_report { overall_health: HEALTHY } } }
W20260926 07:44:24.550623 31366 tablet.cc:2369] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:44:24.556169 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.551196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.11.9.65:37791)
0926 07:44:24.551429 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.551436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.551657 (+ 221us) server_negotiation.cc:407] Connection header received
0926 07:44:24.551815 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.551818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.551910 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.552024 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:44:24.552325 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.552997 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.554362 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.554759 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.554853 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.555269 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:44:24.555728 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:44:24.555835 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.555907 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.555966 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 07:44:24.564059 31581 raft_consensus.cc:1261] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 1 FOLLOWER]: Refusing update from remote peer 575c57a5e9e34c72b8d261ef1abc6e7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:24.564059 31451 raft_consensus.cc:1261] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 1 FOLLOWER]: Refusing update from remote peer 575c57a5e9e34c72b8d261ef1abc6e7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:44:24.565029 31655 consensus_queue.cc:1035] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e81144ce19847f688b90265c659cd03" member_type: VOTER last_known_addr { host: "127.11.9.66" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:24.565472 31652 consensus_queue.cc:1035] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f70a4d395f7f4005beeeedae6b134fc3" member_type: VOTER last_known_addr { host: "127.11.9.67" port: 42227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:44:24.591101 31179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:24.591409 31179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 07:44:24.595103 31179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 467a7fd786814d3eb110715bc9b50ae8: Sending DeleteTablet for 3 replicas of tablet 6e6fe594977840478df1544885366729
I20260926 07:44:24.596155 31431 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6fe594977840478df1544885366729 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:24 UTC) from {username='slave'} at 127.0.0.1:56524
I20260926 07:44:24.596323 31300 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6fe594977840478df1544885366729 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:24 UTC) from {username='slave'} at 127.0.0.1:55460
I20260926 07:44:24.596939 31561 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6fe594977840478df1544885366729 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:44:24 UTC) from {username='slave'} at 127.0.0.1:49648
I20260926 07:44:24.597580 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31238
I20260926 07:44:24.599761 31668 tablet_replica.cc:330] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: stopping tablet replica
I20260926 07:44:24.600246 31668 raft_consensus.cc:2229] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:24.600528 31670 tablet_replica.cc:330] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: stopping tablet replica
I20260926 07:44:24.600903 31668 raft_consensus.cc:2258] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:24.600903 31670 raft_consensus.cc:2229] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:44:24.601361 31670 raft_consensus.cc:2258] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:24.601752 31669 tablet_replica.cc:330] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: stopping tablet replica
I20260926 07:44:24.602156 31669 raft_consensus.cc:2229] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:44:24.602839 31669 raft_consensus.cc:2258] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:44:24.603672 31670 ts_tablet_manager.cc:1905] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:24.606988 31668 ts_tablet_manager.cc:1905] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:24.608268 31669 ts_tablet_manager.cc:1905] T 6e6fe594977840478df1544885366729 P 575c57a5e9e34c72b8d261ef1abc6e7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:44:24.610562 31670 ts_tablet_manager.cc:1918] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:24.610715 31670 log.cc:1196] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-2/wal/wals/6e6fe594977840478df1544885366729
I20260926 07:44:24.611208 31670 ts_tablet_manager.cc:1939] T 6e6fe594977840478df1544885366729 P f70a4d395f7f4005beeeedae6b134fc3: Deleting consensus metadata
I20260926 07:44:24.612972 31164 catalog_manager.cc:4841] TS f70a4d395f7f4005beeeedae6b134fc3 (127.11.9.67:42227): tablet 6e6fe594977840478df1544885366729 (table test-table [id=63154adf0dfd449f8e12d1a6c3554c90]) successfully deleted
I20260926 07:44:24.609138 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.598906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.11.9.65:37791)
0926 07:44:24.599071 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.599077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.599939 (+ 862us) server_negotiation.cc:407] Connection header received
0926 07:44:24.600108 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.600113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.600227 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.600340 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:24.602066 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.603342 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.605009 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.605781 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.605925 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.607446 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:24.607456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:24.607471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:24.608215 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:24.608282 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:24.608312 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:24.608402 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.608648 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.608745 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:44:24.614080 31360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:24.615284 31668 ts_tablet_manager.cc:1918] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:44:24.615485 31668 log.cc:1196] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Deleting WAL directory at /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790408351541924-11301-0/minicluster-data/ts-1/wal/wals/6e6fe594977840478df1544885366729
I20260926 07:44:24.616226 31668 ts_tablet_manager.cc:1939] T 6e6fe594977840478df1544885366729 P 3e81144ce19847f688b90265c659cd03: Deleting consensus metadata
I20260926 07:44:24.618726 31164 catalog_manager.cc:4841] TS 3e81144ce19847f688b90265c659cd03 (127.11.9.66:34917): tablet 6e6fe594977840478df1544885366729 (table test-table [id=63154adf0dfd449f8e12d1a6c3554c90]) successfully deleted
I20260926 07:44:24.711032 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31238
I20260926 07:44:24.729344 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31369
W20260926 07:44:24.729535 31164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.9.65:37791 (error 108)
I20260926 07:44:24.731410 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.730953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56634)
0926 07:44:24.731189 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.731282 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260926 07:44:24.731621 31164 catalog_manager.cc:4584] TS 575c57a5e9e34c72b8d261ef1abc6e7e (127.11.9.65:37791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e6fe594977840478df1544885366729: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
I20260926 07:44:24.739312 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.730112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.11.9.66:34917)
0926 07:44:24.730296 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.730302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.730464 (+ 162us) server_negotiation.cc:407] Connection header received
0926 07:44:24.730673 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.730678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.730781 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.730910 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:24.731328 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.732177 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.733498 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.733896 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.734024 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.737936 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:24.737945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:24.737964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:24.738544 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:24.738596 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:24.738616 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:24.738706 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.738861 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.738951 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 07:44:24.739925 31491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:24.784931 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.784493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56646)
0926 07:44:24.784688 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.784790 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 07:44:24.786006 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.785667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56660)
0926 07:44:24.785820 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.785898 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 07:44:24.833235 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31369
I20260926 07:44:24.846801 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.846195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56662)
0926 07:44:24.846450 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.846572 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 07:44:24.847930 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.847510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56672)
0926 07:44:24.847682 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.847777 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 07:44:24.850419 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31499
I20260926 07:44:24.857401 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.851272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.11.9.67:42227)
0926 07:44:24.851467 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:24.851476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:24.851601 (+ 125us) server_negotiation.cc:407] Connection header received
0926 07:44:24.851859 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:24.851866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:24.851994 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:24.852101 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:24.852622 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.853381 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.854731 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:24.855096 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:24.855182 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:24.856160 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:24.856167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:24.856181 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:24.856727 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:24.856770 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:24.856788 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:24.856868 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:24.857000 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 07:44:24.857081 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:44:24.858031 31621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:24.928735 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.928230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56682)
0926 07:44:24.928467 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.928564 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260926 07:44:24.929857 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:24.929485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.65:37791 (local address 127.0.0.1:56684)
0926 07:44:24.929680 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 07:44:24.929750 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.65:37791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 07:44:24.946419 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31499
I20260926 07:44:24.963536 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31148
I20260926 07:44:24.964720 31209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:25.073652 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31148
2026-09-26T07:44:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2826 ms)
[----------] 6 tests from EncryptionPolicyTest (22040 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:25 dist-test-slave-x492 krb5kdc[31679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:25 dist-test-slave-x492 krb5kdc[31679](info): set up 2 sockets
Sep 26 07:44:25 dist-test-slave-x492 krb5kdc[31679](info): commencing operation
krb5kdc: starting...
W20260926 07:44:26.213847 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:26 dist-test-slave-x492 krb5kdc[31679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:26.460362 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:26.789803 31692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:26.790069 31692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:26.790258 31692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:26.797611 31692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:26.797712 31692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:26.797825 31692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:26.797932 31692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:26.798013 31692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:26.807881 31692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:26.809130 31692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:26.810624 31692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:26.822151 31700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:26.822180 31698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:26.822742 31697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:26.828500 31692 server_base.cc:1034] running on GCE node
I20260926 07:44:26.829376 31692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:26.830734 31692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:26.831964 31692 hybrid_clock.cc:648] HybridClock initialized: now 1790408666831928 us; error 46 us; skew 500 ppm
I20260926 07:44:26.832316 31692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:26 dist-test-slave-x492 krb5kdc[31679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:26.835429 31692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:26.836897 31692 webserver.cc:460] Webserver started at http://127.0.0.1:37203/ using document root <none> and password file <none>
I20260926 07:44:26.837643 31692 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:26.837795 31692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:26.838101 31692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:26.840984 31692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "ffc8e4954526482cb079f57813a3a30b"
format_stamp: "Formatted at 2026-09-26 07:44:26 on dist-test-slave-x492"
server_key: "5bd66e86d05b7d054a11bda6da13324d"
server_key_iv: "6c8491c9a474ed6e245e29ebc2bcbd20"
server_key_version: "encryptionkey@0"
I20260926 07:44:26.841806 31692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "ffc8e4954526482cb079f57813a3a30b"
format_stamp: "Formatted at 2026-09-26 07:44:26 on dist-test-slave-x492"
server_key: "5bd66e86d05b7d054a11bda6da13324d"
server_key_iv: "6c8491c9a474ed6e245e29ebc2bcbd20"
server_key_version: "encryptionkey@0"
I20260926 07:44:26.846791 31692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:44:26.850109 31707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:26.851382 31692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:26.851560 31692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "ffc8e4954526482cb079f57813a3a30b"
format_stamp: "Formatted at 2026-09-26 07:44:26 on dist-test-slave-x492"
server_key: "5bd66e86d05b7d054a11bda6da13324d"
server_key_iv: "6c8491c9a474ed6e245e29ebc2bcbd20"
server_key_version: "encryptionkey@0"
I20260926 07:44:26.851753 31692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:26.867118 31692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:26.871021 31692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:26.871312 31692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:26.890165 31692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38971
I20260926 07:44:26.890199 31758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38971 every 8 connection(s)
I20260926 07:44:26.891533 31692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:26.894955 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31692
I20260926 07:44:26.895190 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:26.895628 31759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:26.896035 11301 external_mini_cluster.cc:1428] Setting key 71fc44acfa71572f603b978cf0391867
I20260926 07:44:26.908761 31759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b: Bootstrap starting.
Sep 26 07:44:26 dist-test-slave-x492 krb5kdc[31679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 07:44:26.912554 31759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:26.913787 31759 log.cc:824] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:26.917109 31759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b: No bootstrap required, opened a new log
I20260926 07:44:26.919203 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:26.898011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57269 (local address 127.0.0.1:38971)
0926 07:44:26.898872 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:26.898883 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:26.898926 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:44:26.900694 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:26.900715 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:26.901130 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:26.901610 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:26.903128 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:26.904383 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:26.905706 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:26.906393 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:26.906613 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:26.911329 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:26.911337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:26.911350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:26.911388 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:26.914956 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:26.915949 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:26.915955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:26.915960 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:26.916110 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:26.916586 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:26.916589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:26.916591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:26.917068 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:26.917484 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:26.917973 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 07:44:26.918250 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":375,"threads_started":1}
I20260926 07:44:26.923018 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:26.925117 31759 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } }
I20260926 07:44:26.925508 31759 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:26.925639 31759 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc8e4954526482cb079f57813a3a30b, State: Initialized, Role: FOLLOWER
I20260926 07:44:26.926321 31759 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } }
I20260926 07:44:26.926584 31759 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:26.926717 31759 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:26.926898 31759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:26.930058 31759 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } }
I20260926 07:44:26.930640 31759 leader_election.cc:302] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc8e4954526482cb079f57813a3a30b; no voters:
I20260926 07:44:26.931365 31759 leader_election.cc:288] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:26.931474 31769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:26.933398 31769 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [term 1 LEADER]: Becoming Leader. State: Replica: ffc8e4954526482cb079f57813a3a30b, State: Running, Role: LEADER
I20260926 07:44:26.933944 31759 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:26.934044 31769 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } }
I20260926 07:44:26.939707 31770 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc8e4954526482cb079f57813a3a30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } } }
I20260926 07:44:26.940243 31770 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:26.940992 31771 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc8e4954526482cb079f57813a3a30b. Latest consensus state: current_term: 1 leader_uuid: "ffc8e4954526482cb079f57813a3a30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc8e4954526482cb079f57813a3a30b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38971 } } }
I20260926 07:44:26.941414 31771 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:26.946771 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:26.927729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55731 (local address 127.0.0.1:38971)
0926 07:44:26.927943 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:26.927952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:26.928420 (+ 468us) server_negotiation.cc:407] Connection header received
0926 07:44:26.928618 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:26.928626 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:26.928687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:26.928764 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:26.930733 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:26.931657 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:26.933148 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:26.933852 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:26.934008 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:26.936895 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:26.936900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:26.936903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:26.936931 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:26.940718 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:26.942758 (+ 2040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:26.942767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:26.942773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:26.942949 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:26.943632 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:26.943640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:26.943646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:26.944188 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:26.944562 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:26.946011 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0926 07:44:26.946260 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:44:26.950177 31778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:26.955448 31778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:26.964572 31778 catalog_manager.cc:1347] Generated new cluster ID: 82d54ade1b9e45db9e2794403faacbab
I20260926 07:44:26.964686 31778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:26.999112 31778 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:27.000991 31778 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:27.010854 31778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc8e4954526482cb079f57813a3a30b: Generated new TSK 0
I20260926 07:44:27.011916 31778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:27.023762 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31692
I20260926 07:44:27.025856 31754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:27.118690 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31692
W20260926 07:44:27.136307 31677 connection.cc:535] client connection to 127.0.0.1:38971 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38971): Connection reset by peer (error 104)
2026-09-26T07:44:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:27 dist-test-slave-x492 krb5kdc[31791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:27 dist-test-slave-x492 krb5kdc[31791](info): set up 2 sockets
Sep 26 07:44:27 dist-test-slave-x492 krb5kdc[31791](info): commencing operation
krb5kdc: starting...
W20260926 07:44:28.225623 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:28 dist-test-slave-x492 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:28.468432 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:28.792595 31804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:28.792953 31804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:28.793133 31804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:28.800312 31804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:28.800473 31804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:28.800588 31804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:28.800688 31804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:28.800773 31804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:28.810547 31804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:28.811838 31804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:28.813139 31804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:28.825088 31810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:28.825217 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:28.825182 31812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:28.826383 31804 server_base.cc:1034] running on GCE node
I20260926 07:44:28.827216 31804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:28.828655 31804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:28.829870 31804 hybrid_clock.cc:648] HybridClock initialized: now 1790408668829827 us; error 49 us; skew 500 ppm
I20260926 07:44:28.830224 31804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:28 dist-test-slave-x492 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:28.833484 31804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:28.835038 31804 webserver.cc:460] Webserver started at http://127.0.0.1:42083/ using document root <none> and password file <none>
I20260926 07:44:28.835744 31804 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:28.835855 31804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:28.836089 31804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:28.838990 31804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "bd016a3d2f7f491f9d04d60843c75279"
format_stamp: "Formatted at 2026-09-26 07:44:28 on dist-test-slave-x492"
server_key: "34135bed53850a19ec507cbf516dc6fe"
server_key_iv: "dc413a7876c8eb70241baa71237cd674"
server_key_version: "encryptionkey@0"
I20260926 07:44:28.839691 31804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "bd016a3d2f7f491f9d04d60843c75279"
format_stamp: "Formatted at 2026-09-26 07:44:28 on dist-test-slave-x492"
server_key: "34135bed53850a19ec507cbf516dc6fe"
server_key_iv: "dc413a7876c8eb70241baa71237cd674"
server_key_version: "encryptionkey@0"
I20260926 07:44:28.844475 31804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 07:44:28.847959 31819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:28.849413 31804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:28.849611 31804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "bd016a3d2f7f491f9d04d60843c75279"
format_stamp: "Formatted at 2026-09-26 07:44:28 on dist-test-slave-x492"
server_key: "34135bed53850a19ec507cbf516dc6fe"
server_key_iv: "dc413a7876c8eb70241baa71237cd674"
server_key_version: "encryptionkey@0"
I20260926 07:44:28.849797 31804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:28.868024 31804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:28.872433 31804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:28.872727 31804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:28.891568 31804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39333
I20260926 07:44:28.891667 31870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39333 every 8 connection(s)
I20260926 07:44:28.892927 31804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:28.896567 31871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:28.901021 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31804
I20260926 07:44:28.901223 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:28.901777 11301 external_mini_cluster.cc:1428] Setting key 1e3971c779af2033c67a56957b47ecd4
I20260926 07:44:28.907879 31871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279: Bootstrap starting.
I20260926 07:44:28.911428 31871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:28.912681 31871 log.cc:824] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:28.915719 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:28.903512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58863 (local address 127.0.0.1:39333)
0926 07:44:28.904301 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:28.904313 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:28.904352 (+ 39us) server_negotiation.cc:407] Connection header received
0926 07:44:28.906464 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:28.906485 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:28.906857 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:28.907162 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:28.907731 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:28.909069 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:28.910977 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:28.911724 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:28.911863 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:28.913120 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:28.913127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:28.913139 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:28.913991 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:28.914086 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:28.914122 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:28.914233 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:28.914549 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 07:44:28.914749 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":372,"threads_started":1}
I20260926 07:44:28.915915 31871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279: No bootstrap required, opened a new log
I20260926 07:44:28.919405 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:28.925675 31871 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } }
I20260926 07:44:28.926168 31871 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:28.926303 31871 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd016a3d2f7f491f9d04d60843c75279, State: Initialized, Role: FOLLOWER
I20260926 07:44:28.926965 31871 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } }
I20260926 07:44:28.927273 31871 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:28.927424 31871 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:28.927644 31871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:28.930591 31871 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } }
I20260926 07:44:28.931257 31871 leader_election.cc:302] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd016a3d2f7f491f9d04d60843c75279; no voters:
I20260926 07:44:28.931671 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:28.924434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40159 (local address 127.0.0.1:39333)
0926 07:44:28.924642 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:28.924649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:28.925196 (+ 547us) server_negotiation.cc:407] Connection header received
0926 07:44:28.925392 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:28.925397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:28.925478 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:28.925605 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:28.926024 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:28.927331 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:28.928780 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:28.929630 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:28.929780 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:28.930847 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:28.930854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:28.930859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:28.930882 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:28.930932 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:28.930956 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:28.931086 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:28.931272 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 07:44:28.931327 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:44:28.931946 31871 leader_election.cc:288] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:28.932404 31881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:28.934178 31871 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:28.939074 31881 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [term 1 LEADER]: Becoming Leader. State: Replica: bd016a3d2f7f491f9d04d60843c75279, State: Running, Role: LEADER
I20260926 07:44:28.939626 31881 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } }
I20260926 07:44:28.942651 31882 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd016a3d2f7f491f9d04d60843c75279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } } }
I20260926 07:44:28.943063 31882 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:28.943552 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd016a3d2f7f491f9d04d60843c75279. Latest consensus state: current_term: 1 leader_uuid: "bd016a3d2f7f491f9d04d60843c75279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd016a3d2f7f491f9d04d60843c75279" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39333 } } }
I20260926 07:44:28.943857 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:28.944048 31895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:28.948557 31895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:28.957661 31895 catalog_manager.cc:1347] Generated new cluster ID: c21c9c9e19e14f03b4b316af2b7f7ff2
I20260926 07:44:28.957803 31895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:28.970196 31895 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:28.971561 31895 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:28.983968 31895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd016a3d2f7f491f9d04d60843c75279: Generated new TSK 0
I20260926 07:44:28.984848 31895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:28.995082 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31804
W20260926 07:44:28.996111 31866 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58863
W20260926 07:44:28.996924 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31804: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:28.997136 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31804
W20260926 07:44:29.010905 31789 connection.cc:535] client connection to 127.0.0.1:39333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39333): Connection reset by peer (error 104)
2026-09-26T07:44:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1874 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:29 dist-test-slave-x492 krb5kdc[31902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:29 dist-test-slave-x492 krb5kdc[31902](info): set up 2 sockets
Sep 26 07:44:29 dist-test-slave-x492 krb5kdc[31902](info): commencing operation
krb5kdc: starting...
W20260926 07:44:30.097582 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:30 dist-test-slave-x492 krb5kdc[31902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:30.340651 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:30.663293 31915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:30.663664 31915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:30.663829 31915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:30.670895 31915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:30.671046 31915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:30.671157 31915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:30.671267 31915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:30.671355 31915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:30.680982 31915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35313
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:30.682355 31915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:30.683691 31915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:30.695235 31920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:30.695267 31921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:30.695381 31923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:30.696409 31915 server_base.cc:1034] running on GCE node
I20260926 07:44:30.697212 31915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:30.698577 31915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:30.699787 31915 hybrid_clock.cc:648] HybridClock initialized: now 1790408670699746 us; error 38 us; skew 500 ppm
I20260926 07:44:30.700132 31915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:30 dist-test-slave-x492 krb5kdc[31902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:30.709082 31915 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:30.710825 31915 webserver.cc:460] Webserver started at http://127.0.0.1:37631/ using document root <none> and password file <none>
I20260926 07:44:30.711555 31915 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:30.711691 31915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:30.712016 31915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:30.715113 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "15d95822aca24f969ab75546c623bda8"
format_stamp: "Formatted at 2026-09-26 07:44:30 on dist-test-slave-x492"
server_key: "5d3c9ba112f6a14dbae8fd25d3b928bf"
server_key_iv: "7124a1685da844982630641e77e86cd4"
server_key_version: "encryptionkey@0"
I20260926 07:44:30.715912 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "15d95822aca24f969ab75546c623bda8"
format_stamp: "Formatted at 2026-09-26 07:44:30 on dist-test-slave-x492"
server_key: "5d3c9ba112f6a14dbae8fd25d3b928bf"
server_key_iv: "7124a1685da844982630641e77e86cd4"
server_key_version: "encryptionkey@0"
I20260926 07:44:30.721372 31915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 07:44:30.724788 31930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:30.726174 31915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:30.726342 31915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "15d95822aca24f969ab75546c623bda8"
format_stamp: "Formatted at 2026-09-26 07:44:30 on dist-test-slave-x492"
server_key: "5d3c9ba112f6a14dbae8fd25d3b928bf"
server_key_iv: "7124a1685da844982630641e77e86cd4"
server_key_version: "encryptionkey@0"
I20260926 07:44:30.726518 31915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:30.732203 31915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:30.736397 31915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:30.736781 31915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:30.755035 31915 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35313
I20260926 07:44:30.755124 31981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35313 every 8 connection(s)
I20260926 07:44:30.756385 31915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:30.759999 31982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:30.763787 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 31915
I20260926 07:44:30.763963 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:30.764497 11301 external_mini_cluster.cc:1428] Setting key 7716b18b38dc8b6790c2d70ff9930295
I20260926 07:44:30.771867 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:30.766225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35843 (local address 127.0.0.1:35313)
0926 07:44:30.766979 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:30.766990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:30.767039 (+ 49us) server_negotiation.cc:407] Connection header received
0926 07:44:30.768642 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:30.768663 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:30.768991 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:30.769392 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:30.770411 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:30.770417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:30.770429 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:30.770443 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:30.770501 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:30.770520 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:30.770688 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:30.770791 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 07:44:30.770996 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":364,"threads_started":1}
I20260926 07:44:30.772737 31982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8: Bootstrap starting.
I20260926 07:44:30.775468 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:30.776237 31982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:30.777480 31982 log.cc:824] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:30.780791 31982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8: No bootstrap required, opened a new log
I20260926 07:44:30.783731 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:30.780634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54401 (local address 127.0.0.1:35313)
0926 07:44:30.780821 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:30.780829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:30.781217 (+ 388us) server_negotiation.cc:407] Connection header received
0926 07:44:30.781433 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:30.781441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:30.781485 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:30.781597 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:30.782616 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:30.782621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:30.782625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:30.782639 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:30.782678 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:30.782693 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:30.782861 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:30.783143 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 07:44:30.783237 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 07:44:30.789642 31982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } }
I20260926 07:44:30.790077 31982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:30.790212 31982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d95822aca24f969ab75546c623bda8, State: Initialized, Role: FOLLOWER
I20260926 07:44:30.790808 31982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } }
I20260926 07:44:30.791067 31982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:30.791250 31982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:30.791447 31982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:30.793874 31982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } }
I20260926 07:44:30.794389 31982 leader_election.cc:302] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15d95822aca24f969ab75546c623bda8; no voters:
I20260926 07:44:30.795048 31982 leader_election.cc:288] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:30.795145 31992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:30.795877 31992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [term 1 LEADER]: Becoming Leader. State: Replica: 15d95822aca24f969ab75546c623bda8, State: Running, Role: LEADER
I20260926 07:44:30.796437 31992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } }
I20260926 07:44:30.797291 31982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:30.799423 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15d95822aca24f969ab75546c623bda8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } } }
I20260926 07:44:30.799623 31994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15d95822aca24f969ab75546c623bda8. Latest consensus state: current_term: 1 leader_uuid: "15d95822aca24f969ab75546c623bda8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d95822aca24f969ab75546c623bda8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35313 } } }
I20260926 07:44:30.800176 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:30.800233 31994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:30.811946 32000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:30.816746 32000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:30.825315 32000 catalog_manager.cc:1347] Generated new cluster ID: 6b5df72d6ea84b07a8254552be8b786c
I20260926 07:44:30.825429 32000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:30.847711 32000 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:30.848188 32000 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:30.855057 32000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15d95822aca24f969ab75546c623bda8: Generated new TSK 0
I20260926 07:44:30.856091 32000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:30.862118 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31915
W20260926 07:44:30.863449 31977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35843
W20260926 07:44:30.864244 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 31915: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:30.864414 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 31915
2026-09-26T07:44:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1870 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:30 dist-test-slave-x492 krb5kdc[32013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:30 dist-test-slave-x492 krb5kdc[32013](info): set up 2 sockets
Sep 26 07:44:30 dist-test-slave-x492 krb5kdc[32013](info): commencing operation
krb5kdc: starting...
W20260926 07:44:31.997718 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:32.240353 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:32.563617 32026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:32.563882 32026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:32.564062 32026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:32.571297 32026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:32.571421 32026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:32.571532 32026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:32.571636 32026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:32.571729 32026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:32.581454 32026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:32.582764 32026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:32.584117 32026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:32.596041 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:32.596104 32034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:32.596269 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:32.596946 32026 server_base.cc:1034] running on GCE node
I20260926 07:44:32.597738 32026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:32.599011 32026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:32.600214 32026 hybrid_clock.cc:648] HybridClock initialized: now 1790408672600175 us; error 25 us; skew 500 ppm
I20260926 07:44:32.600553 32026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:32.603639 32026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:32.605084 32026 webserver.cc:460] Webserver started at http://127.0.0.1:34043/ using document root <none> and password file <none>
I20260926 07:44:32.605758 32026 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:32.605901 32026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:32.606191 32026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:32.609004 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "bfac2d3dd2b64cffa3980f5416e6564d"
format_stamp: "Formatted at 2026-09-26 07:44:32 on dist-test-slave-x492"
server_key: "943f2bf94b0e75194346a414676f8c66"
server_key_iv: "eba91a3a8f7352ac0d0eb7ba589b6c19"
server_key_version: "encryptionkey@0"
I20260926 07:44:32.609817 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "bfac2d3dd2b64cffa3980f5416e6564d"
format_stamp: "Formatted at 2026-09-26 07:44:32 on dist-test-slave-x492"
server_key: "943f2bf94b0e75194346a414676f8c66"
server_key_iv: "eba91a3a8f7352ac0d0eb7ba589b6c19"
server_key_version: "encryptionkey@0"
I20260926 07:44:32.614683 32026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:44:32.617937 32041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:32.619112 32026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:32.619288 32026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "bfac2d3dd2b64cffa3980f5416e6564d"
format_stamp: "Formatted at 2026-09-26 07:44:32 on dist-test-slave-x492"
server_key: "943f2bf94b0e75194346a414676f8c66"
server_key_iv: "eba91a3a8f7352ac0d0eb7ba589b6c19"
server_key_version: "encryptionkey@0"
I20260926 07:44:32.619472 32026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:32.640249 32026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:32.644068 32026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:32.644349 32026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:32.662215 32026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44281
I20260926 07:44:32.662254 32092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44281 every 8 connection(s)
I20260926 07:44:32.663554 32026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:32.667177 32093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:32.672854 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32026
I20260926 07:44:32.673029 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:32.673520 11301 external_mini_cluster.cc:1428] Setting key be1501d361245f33696c8e3e4d45a64c
I20260926 07:44:32.677874 32093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d: Bootstrap starting.
I20260926 07:44:32.681542 32093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:32.682839 32093 log.cc:824] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:32.686048 32093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d: No bootstrap required, opened a new log
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 07:44:32.696527 32093 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } }
I20260926 07:44:32.696965 32093 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:32.697074 32093 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfac2d3dd2b64cffa3980f5416e6564d, State: Initialized, Role: FOLLOWER
I20260926 07:44:32.697579 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:32.675259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43591 (local address 127.0.0.1:44281)
0926 07:44:32.676065 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:32.676076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:32.676117 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:44:32.677779 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:32.677801 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:32.678216 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:32.678531 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:32.680348 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:32.681728 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:32.683539 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:32.684433 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:32.684556 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:32.689182 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:32.689194 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:32.689216 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:32.689269 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:32.693191 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:32.693853 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:32.693864 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:32.693875 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:32.694105 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:32.694463 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:32.694468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:32.694472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:32.695165 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:32.695453 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:32.695754 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 07:44:32.696172 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":405,"threads_started":1}
I20260926 07:44:32.697798 32093 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } }
I20260926 07:44:32.698099 32093 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:32.698264 32093 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:32.698454 32093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:32.700995 32093 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } }
I20260926 07:44:32.701363 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:32.701565 32093 leader_election.cc:302] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfac2d3dd2b64cffa3980f5416e6564d; no voters:
I20260926 07:44:32.702143 32093 leader_election.cc:288] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:32.702247 32098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:32.703792 32098 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [term 1 LEADER]: Becoming Leader. State: Replica: bfac2d3dd2b64cffa3980f5416e6564d, State: Running, Role: LEADER
I20260926 07:44:32.704352 32098 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } }
I20260926 07:44:32.704559 32093 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:32.708509 32099 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } } }
I20260926 07:44:32.708963 32099 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:32.709625 32100 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfac2d3dd2b64cffa3980f5416e6564d. Latest consensus state: current_term: 1 leader_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfac2d3dd2b64cffa3980f5416e6564d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44281 } } }
I20260926 07:44:32.709997 32100 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:32.717643 32109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:32.723316 32109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:32.734684 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:32.707632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53227 (local address 127.0.0.1:44281)
0926 07:44:32.707803 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:32.707810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:32.715825 (+ 8015us) server_negotiation.cc:407] Connection header received
0926 07:44:32.716804 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:32.716808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:32.716872 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:32.717024 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:32.719180 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:32.720506 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:32.722581 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:32.723293 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:32.723433 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:32.726888 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:32.726895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:32.726901 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:32.726958 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:32.731016 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:32.731922 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:32.731928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:32.731932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:32.732100 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:32.732567 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:32.732573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:32.732577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:32.733044 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:32.733370 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:32.734042 (+ 672us) server_negotiation.cc:299] Negotiation successful
0926 07:44:32.734284 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:44:32.739943 32109 catalog_manager.cc:1347] Generated new cluster ID: d7d08667204d41df983187e129e6a4ae
I20260926 07:44:32.740058 32109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:32.773368 32109 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:32.774717 32109 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:32.780893 32109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfac2d3dd2b64cffa3980f5416e6564d: Generated new TSK 0
I20260926 07:44:32.781826 32109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:32.797770 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32026
I20260926 07:44:32.799072 32088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:32.891330 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32026
2026-09-26T07:44:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32125](info): set up 2 sockets
Sep 26 07:44:32 dist-test-slave-x492 krb5kdc[32125](info): commencing operation
krb5kdc: starting...
W20260926 07:44:33.993541 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:34 dist-test-slave-x492 krb5kdc[32125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:34.236711 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:34.558996 32138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:34.559342 32138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:34.559559 32138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:34.566672 32138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:34.566823 32138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:34.566936 32138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:34.567054 32138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:34.567140 32138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:34.576834 32138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:34.578158 32138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:34.579443 32138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:34.591261 32144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:34.591338 32143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:34.591497 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:34.592687 32138 server_base.cc:1034] running on GCE node
I20260926 07:44:34.593487 32138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:34.594807 32138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:34.596004 32138 hybrid_clock.cc:648] HybridClock initialized: now 1790408674595963 us; error 30 us; skew 500 ppm
I20260926 07:44:34.596359 32138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:34 dist-test-slave-x492 krb5kdc[32125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:34.599442 32138 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:34.600872 32138 webserver.cc:460] Webserver started at http://127.0.0.1:42453/ using document root <none> and password file <none>
I20260926 07:44:34.601583 32138 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:34.601719 32138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:34.602033 32138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:34.604846 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "1a76c5b6165642f9b5403c19651c1735"
format_stamp: "Formatted at 2026-09-26 07:44:34 on dist-test-slave-x492"
server_key: "12bb3fa9e936e9c153aeed6c7f44f223"
server_key_iv: "215464bf3d4037d779b49dd33459b964"
server_key_version: "encryptionkey@0"
I20260926 07:44:34.605623 32138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "1a76c5b6165642f9b5403c19651c1735"
format_stamp: "Formatted at 2026-09-26 07:44:34 on dist-test-slave-x492"
server_key: "12bb3fa9e936e9c153aeed6c7f44f223"
server_key_iv: "215464bf3d4037d779b49dd33459b964"
server_key_version: "encryptionkey@0"
I20260926 07:44:34.610446 32138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 07:44:34.613730 32153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:34.614921 32138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:34.615106 32138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "1a76c5b6165642f9b5403c19651c1735"
format_stamp: "Formatted at 2026-09-26 07:44:34 on dist-test-slave-x492"
server_key: "12bb3fa9e936e9c153aeed6c7f44f223"
server_key_iv: "215464bf3d4037d779b49dd33459b964"
server_key_version: "encryptionkey@0"
I20260926 07:44:34.615289 32138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:34.647322 32138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:34.651692 32138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:34.652009 32138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:34.671579 32138 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39527
I20260926 07:44:34.671617 32204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39527 every 8 connection(s)
I20260926 07:44:34.673027 32138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:34.676638 32205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:34.680259 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32138
I20260926 07:44:34.680423 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:34.680961 11301 external_mini_cluster.cc:1428] Setting key 38911583c31cc3eb7984c746556ed809
I20260926 07:44:34.688910 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735: Bootstrap starting.
I20260926 07:44:34.692502 32205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:34.693661 32205 log.cc:824] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:34.694226 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:34.682760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55589 (local address 127.0.0.1:39527)
0926 07:44:34.683503 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:34.683514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:34.683554 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:44:34.685243 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:34.685265 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:34.685688 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:34.686013 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:34.686558 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:34.687856 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:34.689622 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:34.690285 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:34.690419 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:34.691673 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:34.691680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:34.691692 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:34.692198 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:34.692268 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:34.692290 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:34.692372 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:34.692728 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 07:44:34.693056 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":340,"threads_started":1}
I20260926 07:44:34.696502 32205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735: No bootstrap required, opened a new log
I20260926 07:44:34.698339 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:34.706765 32205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260926 07:44:34.707154 32205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:34.707275 32205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a76c5b6165642f9b5403c19651c1735, State: Initialized, Role: FOLLOWER
I20260926 07:44:34.707854 32205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260926 07:44:34.708143 32205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:34.708309 32205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:34.708496 32205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:34.710070 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:34.703399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38613 (local address 127.0.0.1:39527)
0926 07:44:34.703615 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:34.703621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:34.704011 (+ 390us) server_negotiation.cc:407] Connection header received
0926 07:44:34.704174 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:34.704179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:34.704266 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:34.704377 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:34.704866 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:34.706044 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:34.707509 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:34.708038 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:34.708187 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:34.709319 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:34.709324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:34.709327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:34.709345 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:34.709384 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:34.709398 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:34.709494 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:34.709621 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 07:44:34.709695 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:44:34.711694 32205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260926 07:44:34.712209 32205 leader_election.cc:302] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a76c5b6165642f9b5403c19651c1735; no voters:
I20260926 07:44:34.712791 32205 leader_election.cc:288] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:34.712947 32215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:34.713716 32215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [term 1 LEADER]: Becoming Leader. State: Replica: 1a76c5b6165642f9b5403c19651c1735, State: Running, Role: LEADER
I20260926 07:44:34.714268 32215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } }
I20260926 07:44:34.714936 32205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:34.724987 32217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a76c5b6165642f9b5403c19651c1735. Latest consensus state: current_term: 1 leader_uuid: "1a76c5b6165642f9b5403c19651c1735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } } }
I20260926 07:44:34.724900 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a76c5b6165642f9b5403c19651c1735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a76c5b6165642f9b5403c19651c1735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39527 } } }
I20260926 07:44:34.725378 32217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:34.725414 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:34.726205 32230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:34.731189 32230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:34.740104 32230 catalog_manager.cc:1347] Generated new cluster ID: 183cb651fdc64354b755513cfa4eb733
I20260926 07:44:34.740213 32230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:34.754087 32230 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:34.755230 32230 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:34.765607 32230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a76c5b6165642f9b5403c19651c1735: Generated new TSK 0
I20260926 07:44:34.766357 32230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:34.772789 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32138
W20260926 07:44:34.773792 32200 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55589
W20260926 07:44:34.774446 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32138: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:34.774580 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32138
2026-09-26T07:44:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1882 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:34 dist-test-slave-x492 krb5kdc[32237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:34 dist-test-slave-x492 krb5kdc[32237](info): set up 2 sockets
Sep 26 07:44:34 dist-test-slave-x492 krb5kdc[32237](info): commencing operation
krb5kdc: starting...
W20260926 07:44:35.873631 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:36 dist-test-slave-x492 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:36.119968 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:36.438091 32250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:36.438374 32250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:36.438556 32250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:36.445583 32250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:36.445714 32250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:36.445820 32250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:36.445926 32250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:36.446010 32250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:36.455484 32250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:36.456751 32250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:36.458052 32250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:36.469676 32255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:36.469702 32256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:36.469635 32258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:36.470736 32250 server_base.cc:1034] running on GCE node
I20260926 07:44:36.471477 32250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:36.472770 32250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:36.473987 32250 hybrid_clock.cc:648] HybridClock initialized: now 1790408676473942 us; error 62 us; skew 500 ppm
I20260926 07:44:36.474326 32250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:36 dist-test-slave-x492 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:36.480536 32250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:36.481884 32250 webserver.cc:460] Webserver started at http://127.0.0.1:45099/ using document root <none> and password file <none>
I20260926 07:44:36.482467 32250 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:36.482573 32250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:36.482806 32250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:36.485612 32250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "b5b10418db1e494b9407a32164015107"
format_stamp: "Formatted at 2026-09-26 07:44:36 on dist-test-slave-x492"
server_key: "4c633bc7865e4317a8be3de055d72c6b"
server_key_iv: "1ba49d818aa4c329922eb9571d5235b1"
server_key_version: "encryptionkey@0"
I20260926 07:44:36.486312 32250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "b5b10418db1e494b9407a32164015107"
format_stamp: "Formatted at 2026-09-26 07:44:36 on dist-test-slave-x492"
server_key: "4c633bc7865e4317a8be3de055d72c6b"
server_key_iv: "1ba49d818aa4c329922eb9571d5235b1"
server_key_version: "encryptionkey@0"
I20260926 07:44:36.491036 32250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:44:36.494166 32265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:36.495464 32250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 07:44:36.495645 32250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "b5b10418db1e494b9407a32164015107"
format_stamp: "Formatted at 2026-09-26 07:44:36 on dist-test-slave-x492"
server_key: "4c633bc7865e4317a8be3de055d72c6b"
server_key_iv: "1ba49d818aa4c329922eb9571d5235b1"
server_key_version: "encryptionkey@0"
I20260926 07:44:36.495827 32250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:36.501215 32250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:36.504858 32250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:36.505139 32250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:36.522739 32250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41677
I20260926 07:44:36.522822 32316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41677 every 8 connection(s)
I20260926 07:44:36.524067 32250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:36.527595 32317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:36.532205 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32250
I20260926 07:44:36.532383 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:36.532858 11301 external_mini_cluster.cc:1428] Setting key 664911edac74693d829417ca7ffd0641
I20260926 07:44:36.538684 32317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107: Bootstrap starting.
I20260926 07:44:36.542135 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:36.534794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48505 (local address 127.0.0.1:41677)
0926 07:44:36.535784 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:36.535799 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:36.536197 (+ 398us) server_negotiation.cc:407] Connection header received
0926 07:44:36.538502 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:36.538529 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:36.538955 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:36.539371 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:36.540289 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:36.540303 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:36.540322 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:36.540346 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:36.540421 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:36.540453 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:36.540663 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:36.540792 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 07:44:36.540982 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":439,"threads_started":1}
I20260926 07:44:36.542558 32317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:36.543792 32317 log.cc:824] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:36.545926 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:36.546696 32317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107: No bootstrap required, opened a new log
I20260926 07:44:36.552433 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:36.550036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39135 (local address 127.0.0.1:41677)
0926 07:44:36.550207 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:36.550214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:36.550594 (+ 380us) server_negotiation.cc:407] Connection header received
0926 07:44:36.550714 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:36.550725 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:36.550784 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:36.550908 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:36.551758 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:36.551764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:36.551769 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:36.551799 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:36.551845 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:36.551864 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:36.552002 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:36.552099 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 07:44:36.552160 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 07:44:36.554733 32317 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } }
I20260926 07:44:36.555075 32317 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:36.555186 32317 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b10418db1e494b9407a32164015107, State: Initialized, Role: FOLLOWER
I20260926 07:44:36.555764 32317 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } }
I20260926 07:44:36.556000 32317 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:36.556144 32317 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:36.556316 32317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:36.558532 32317 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } }
I20260926 07:44:36.559011 32317 leader_election.cc:302] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b10418db1e494b9407a32164015107; no voters:
I20260926 07:44:36.559530 32317 leader_election.cc:288] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:36.559657 32327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:36.560325 32327 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [term 1 LEADER]: Becoming Leader. State: Replica: b5b10418db1e494b9407a32164015107, State: Running, Role: LEADER
I20260926 07:44:36.560845 32327 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } }
I20260926 07:44:36.561574 32317 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:36.563563 32329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b10418db1e494b9407a32164015107. Latest consensus state: current_term: 1 leader_uuid: "b5b10418db1e494b9407a32164015107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } } }
I20260926 07:44:36.564059 32329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:36.564532 32328 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5b10418db1e494b9407a32164015107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b10418db1e494b9407a32164015107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41677 } } }
I20260926 07:44:36.564795 32328 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:36.566529 32336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:36.571395 32336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:36.580083 32336 catalog_manager.cc:1347] Generated new cluster ID: 63685bed193e4e8296afa395df085550
I20260926 07:44:36.580194 32336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:36.595438 32336 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:36.595954 32336 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:36.603114 32336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5b10418db1e494b9407a32164015107: Generated new TSK 0
I20260926 07:44:36.603885 32336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:36.609447 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32250
W20260926 07:44:36.610554 32312 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48505
W20260926 07:44:36.611268 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32250: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:36.611444 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32250
2026-09-26T07:44:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1836 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:36 dist-test-slave-x492 krb5kdc[32348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:36 dist-test-slave-x492 krb5kdc[32348](info): set up 2 sockets
Sep 26 07:44:36 dist-test-slave-x492 krb5kdc[32348](info): commencing operation
krb5kdc: starting...
W20260926 07:44:37.721630 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:37 dist-test-slave-x492 krb5kdc[32348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:37.969337 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:38.312111 32361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:38.312455 32361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:38.312623 32361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:38.319769 32361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:38.319918 32361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:38.320031 32361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:38.320140 32361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:38.320227 32361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:38.329597 32361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:38.330853 32361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:38.332139 32361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:38.344090 32369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:38.344168 32366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:38.344170 32367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:38.345005 32361 server_base.cc:1034] running on GCE node
I20260926 07:44:38.345875 32361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:38.347311 32361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:38.348562 32361 hybrid_clock.cc:648] HybridClock initialized: now 1790408678348522 us; error 60 us; skew 500 ppm
I20260926 07:44:38.348922 32361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:38 dist-test-slave-x492 krb5kdc[32348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:38.352100 32361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:38.353709 32361 webserver.cc:460] Webserver started at http://127.0.0.1:38687/ using document root <none> and password file <none>
I20260926 07:44:38.354393 32361 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:38.354538 32361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:38.354856 32361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:38.357800 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "5b30ca05389d4094b8bf7e0f85c334e0"
format_stamp: "Formatted at 2026-09-26 07:44:38 on dist-test-slave-x492"
server_key: "ed37309731ff10a90bdc1ac35dd15e54"
server_key_iv: "f81491b2da50a0e4a5fb9ce3cc107030"
server_key_version: "encryptionkey@0"
I20260926 07:44:38.358587 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "5b30ca05389d4094b8bf7e0f85c334e0"
format_stamp: "Formatted at 2026-09-26 07:44:38 on dist-test-slave-x492"
server_key: "ed37309731ff10a90bdc1ac35dd15e54"
server_key_iv: "f81491b2da50a0e4a5fb9ce3cc107030"
server_key_version: "encryptionkey@0"
I20260926 07:44:38.363413 32361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 07:44:38.366777 32376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:38.368011 32361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:44:38.368182 32361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "5b30ca05389d4094b8bf7e0f85c334e0"
format_stamp: "Formatted at 2026-09-26 07:44:38 on dist-test-slave-x492"
server_key: "ed37309731ff10a90bdc1ac35dd15e54"
server_key_iv: "f81491b2da50a0e4a5fb9ce3cc107030"
server_key_version: "encryptionkey@0"
I20260926 07:44:38.368395 32361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:38.384160 32361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:38.388649 32361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:38.388947 32361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:38.407280 32361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35345
I20260926 07:44:38.407310 32427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35345 every 8 connection(s)
I20260926 07:44:38.408689 32361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:38.412272 32428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:38.416086 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32361
I20260926 07:44:38.416272 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:38.416818 11301 external_mini_cluster.cc:1428] Setting key c71d1abd1bd53a8321f630e977fb747e
I20260926 07:44:38.424917 32428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: Bootstrap starting.
I20260926 07:44:38.428664 32428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:38.430033 32428 log.cc:824] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: Log is configured to *not* fsync() on all Append() calls
Sep 26 07:44:38 dist-test-slave-x492 krb5kdc[32348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 07:44:38.433290 32428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: No bootstrap required, opened a new log
I20260926 07:44:38.440352 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:38.418561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:53225 (local address 127.0.0.1:35345)
0926 07:44:38.419272 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:38.419283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:38.419331 (+ 48us) server_negotiation.cc:407] Connection header received
0926 07:44:38.420858 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:38.420878 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:38.421333 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:38.421643 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:38.423351 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:38.424608 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:38.426313 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:38.426974 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:38.427101 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:38.431839 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:38.431849 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:38.431869 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:38.431914 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:38.435778 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:38.436533 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:38.436541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:38.436551 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:38.436779 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:38.437316 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:38.437323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:38.437328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:38.438012 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:38.438328 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:38.438686 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 07:44:38.439021 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":336,"threads_started":1}
I20260926 07:44:38.443341 32428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } }
I20260926 07:44:38.443683 32428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:38.443783 32428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b30ca05389d4094b8bf7e0f85c334e0, State: Initialized, Role: FOLLOWER
I20260926 07:44:38.444226 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:38.444444 32428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } }
I20260926 07:44:38.444676 32428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:38.444821 32428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:38.444994 32428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:38.447196 32428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } }
I20260926 07:44:38.447801 32428 leader_election.cc:302] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b30ca05389d4094b8bf7e0f85c334e0; no voters:
I20260926 07:44:38.448442 32428 leader_election.cc:288] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:38.448552 32437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:38.449868 32437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b30ca05389d4094b8bf7e0f85c334e0, State: Running, Role: LEADER
I20260926 07:44:38.450587 32437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } }
I20260926 07:44:38.450752 32428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:38.463197 32438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } } }
I20260926 07:44:38.463581 32438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:38.463150 32440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b30ca05389d4094b8bf7e0f85c334e0. Latest consensus state: current_term: 1 leader_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b30ca05389d4094b8bf7e0f85c334e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35345 } } }
I20260926 07:44:38.464267 32440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:38.465711 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:38.448809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:39197 (local address 127.0.0.1:35345)
0926 07:44:38.449013 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:38.449021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:38.449794 (+ 773us) server_negotiation.cc:407] Connection header received
0926 07:44:38.450064 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:38.450069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:38.450133 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:38.450198 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:38.451696 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:38.453193 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:38.454587 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:38.455420 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:38.455564 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:38.458329 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:38.458346 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:38.458350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:38.458378 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:38.461674 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:38.462833 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:38.462838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:38.462841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:38.463005 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:38.463694 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:38.463699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:38.463706 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:38.464145 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:38.464435 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:38.465109 (+ 674us) server_negotiation.cc:299] Negotiation successful
0926 07:44:38.465345 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":63}
W20260926 07:44:38.468117 32454 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:44:38.468215 32454 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:44:38.468366 32455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:38.470299 32455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:38.479210 32455 catalog_manager.cc:1347] Generated new cluster ID: c82a43d43167439b91caf177434ef811
I20260926 07:44:38.479331 32455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:38.487085 32455 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:38.488744 32455 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:38.495458 32455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b30ca05389d4094b8bf7e0f85c334e0: Generated new TSK 0
I20260926 07:44:38.496354 32455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:38.502868 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32361
I20260926 07:44:38.504107 32423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:38.594552 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32361
W20260926 07:44:38.611246 32346 connection.cc:535] client connection to 127.0.0.1:35345 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35345): Connection reset by peer (error 104)
2026-09-26T07:44:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1982 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:38 dist-test-slave-x492 krb5kdc[32461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:38 dist-test-slave-x492 krb5kdc[32461](info): set up 2 sockets
Sep 26 07:44:38 dist-test-slave-x492 krb5kdc[32461](info): commencing operation
krb5kdc: starting...
W20260926 07:44:39.689692 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:39 dist-test-slave-x492 krb5kdc[32461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:39.932693 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:40.264340 32474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:40.264616 32474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:40.264707 32474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:40.271901 32474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:40.272001 32474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:40.272045 32474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:40.272104 32474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:40.272147 32474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:40.282040 32474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:40.283190 32474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:40.284394 32474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:40.295686 32479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:40.295842 32482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:40.295902 32480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:40.297128 32474 server_base.cc:1034] running on GCE node
I20260926 07:44:40.297946 32474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:40.299326 32474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:40.300568 32474 hybrid_clock.cc:648] HybridClock initialized: now 1790408680300526 us; error 49 us; skew 500 ppm
I20260926 07:44:40.300926 32474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:40 dist-test-slave-x492 krb5kdc[32461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:40.304653 32474 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:40.306326 32474 webserver.cc:460] Webserver started at http://127.0.0.1:37611/ using document root <none> and password file <none>
I20260926 07:44:40.307085 32474 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:40.307250 32474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:40.307578 32474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:40.310968 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "0d12247f30db47c48f193c6523ccaacf"
format_stamp: "Formatted at 2026-09-26 07:44:40 on dist-test-slave-x492"
server_key: "9f5755cc1553fd6d70aa011578c49c3a"
server_key_iv: "669f2cd19ea1f23d6dcf67ed522e3b6e"
server_key_version: "encryptionkey@0"
I20260926 07:44:40.311846 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "0d12247f30db47c48f193c6523ccaacf"
format_stamp: "Formatted at 2026-09-26 07:44:40 on dist-test-slave-x492"
server_key: "9f5755cc1553fd6d70aa011578c49c3a"
server_key_iv: "669f2cd19ea1f23d6dcf67ed522e3b6e"
server_key_version: "encryptionkey@0"
I20260926 07:44:40.317533 32474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 07:44:40.321157 32489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:40.322563 32474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:40.322770 32474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "0d12247f30db47c48f193c6523ccaacf"
format_stamp: "Formatted at 2026-09-26 07:44:40 on dist-test-slave-x492"
server_key: "9f5755cc1553fd6d70aa011578c49c3a"
server_key_iv: "669f2cd19ea1f23d6dcf67ed522e3b6e"
server_key_version: "encryptionkey@0"
I20260926 07:44:40.322988 32474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:40.343889 32474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:40.349030 32474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:40.349380 32474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:40.368089 32474 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36045
I20260926 07:44:40.368115 32540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36045 every 8 connection(s)
I20260926 07:44:40.369606 32474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:40.373286 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:40.377373 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32474
I20260926 07:44:40.377583 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:40.378048 11301 external_mini_cluster.cc:1428] Setting key b57d7fe63f79d7475a802b3f52eeb610
I20260926 07:44:40.384229 32541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf: Bootstrap starting.
I20260926 07:44:40.387950 32541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:40.389236 32541 log.cc:824] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:40.392387 32541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf: No bootstrap required, opened a new log
I20260926 07:44:40.392853 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:40.379929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36503 (local address 127.0.0.1:36045)
0926 07:44:40.380713 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:40.380728 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:40.380771 (+ 43us) server_negotiation.cc:407] Connection header received
0926 07:44:40.382898 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:40.382922 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:40.383328 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:40.383669 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:40.384313 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:40.385883 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:40.387767 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:40.388596 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:40.388752 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:40.390207 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:40.390219 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:40.390241 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:40.390996 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:40.391075 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:40.391101 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:40.391229 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:40.391563 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 07:44:40.391760 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":357,"threads_started":1}
I20260926 07:44:40.396966 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:40.400727 32541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } }
I20260926 07:44:40.401078 32541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:40.401232 32541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d12247f30db47c48f193c6523ccaacf, State: Initialized, Role: FOLLOWER
I20260926 07:44:40.401810 32541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } }
I20260926 07:44:40.402181 32541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:40.402338 32541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:40.402527 32541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:40.405455 32541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } }
I20260926 07:44:40.405927 32541 leader_election.cc:302] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d12247f30db47c48f193c6523ccaacf; no voters:
I20260926 07:44:40.406585 32541 leader_election.cc:288] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:40.406697 32551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:40.407925 32551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [term 1 LEADER]: Becoming Leader. State: Replica: 0d12247f30db47c48f193c6523ccaacf, State: Running, Role: LEADER
I20260926 07:44:40.408602 32551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } }
I20260926 07:44:40.409490 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:40.401875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:46307 (local address 127.0.0.1:36045)
0926 07:44:40.402159 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:40.402170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:40.402675 (+ 505us) server_negotiation.cc:407] Connection header received
0926 07:44:40.402766 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:40.402772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:40.402857 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:40.402960 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:40.403505 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:40.404650 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:40.406215 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:40.407061 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:40.407207 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:40.408616 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:40.408621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:40.408625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:40.408644 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:40.408721 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:40.408746 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:40.408880 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:40.409073 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 07:44:40.409124 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260926 07:44:40.409502 32541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:40.411975 32552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d12247f30db47c48f193c6523ccaacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } } }
I20260926 07:44:40.412366 32552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:40.412062 32553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d12247f30db47c48f193c6523ccaacf. Latest consensus state: current_term: 1 leader_uuid: "0d12247f30db47c48f193c6523ccaacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d12247f30db47c48f193c6523ccaacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36045 } } }
I20260926 07:44:40.412521 32553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:40.414203 32557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:40.419466 32557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:40.430601 32557 catalog_manager.cc:1347] Generated new cluster ID: d46a9bc848854c309d24c1167393f66c
I20260926 07:44:40.430737 32557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:40.475862 32557 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:40.477147 32557 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:40.484171 32557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d12247f30db47c48f193c6523ccaacf: Generated new TSK 0
I20260926 07:44:40.484952 32557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:40.498123 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32474
W20260926 07:44:40.499360 32536 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:36503
W20260926 07:44:40.500041 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32474: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:40.500177 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32474
W20260926 07:44:40.516873 32459 connection.cc:535] client connection to 127.0.0.1:36045 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36045): Connection reset by peer (error 104)
2026-09-26T07:44:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1905 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:40 dist-test-slave-x492 krb5kdc[32572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:40 dist-test-slave-x492 krb5kdc[32572](info): set up 2 sockets
Sep 26 07:44:40 dist-test-slave-x492 krb5kdc[32572](info): commencing operation
krb5kdc: starting...
W20260926 07:44:41.593572 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:41 dist-test-slave-x492 krb5kdc[32572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:41.836849 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:42.169826 32585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:42.170161 32585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:42.170329 32585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:42.177595 32585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:42.177757 32585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:42.177860 32585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:42.177965 32585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:42.178042 32585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:42.188083 32585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:42.189571 32585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:42.190881 32585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:42.202822 32591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:42.202968 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:42.202976 32593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:42.204072 32585 server_base.cc:1034] running on GCE node
I20260926 07:44:42.204903 32585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:42.206339 32585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:42.207588 32585 hybrid_clock.cc:648] HybridClock initialized: now 1790408682207553 us; error 47 us; skew 500 ppm
I20260926 07:44:42.207932 32585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:42 dist-test-slave-x492 krb5kdc[32572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:42.214442 32585 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:42.215957 32585 webserver.cc:460] Webserver started at http://127.0.0.1:37237/ using document root <none> and password file <none>
I20260926 07:44:42.216624 32585 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:42.216776 32585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:42.217085 32585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:42.220057 32585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "e4327ef310bc4b7aa8ab1c44287337dc"
format_stamp: "Formatted at 2026-09-26 07:44:42 on dist-test-slave-x492"
server_key: "968926653c8395f4a877ca33dbd9bdc3"
server_key_iv: "b08e5852ebd8fcba4ba0a0e26bc623d5"
server_key_version: "encryptionkey@0"
I20260926 07:44:42.220845 32585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "e4327ef310bc4b7aa8ab1c44287337dc"
format_stamp: "Formatted at 2026-09-26 07:44:42 on dist-test-slave-x492"
server_key: "968926653c8395f4a877ca33dbd9bdc3"
server_key_iv: "b08e5852ebd8fcba4ba0a0e26bc623d5"
server_key_version: "encryptionkey@0"
I20260926 07:44:42.226192 32585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 07:44:42.229831 32600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:42.231261 32585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:42.231451 32585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "e4327ef310bc4b7aa8ab1c44287337dc"
format_stamp: "Formatted at 2026-09-26 07:44:42 on dist-test-slave-x492"
server_key: "968926653c8395f4a877ca33dbd9bdc3"
server_key_iv: "b08e5852ebd8fcba4ba0a0e26bc623d5"
server_key_version: "encryptionkey@0"
I20260926 07:44:42.231673 32585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:42.237630 32585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:42.241696 32585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:42.241997 32585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:42.260531 32585 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41119
I20260926 07:44:42.260619 32651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41119 every 8 connection(s)
I20260926 07:44:42.261904 32585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:42.265681 32652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:42.270797 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32585
I20260926 07:44:42.270980 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:42.271514 11301 external_mini_cluster.cc:1428] Setting key bca30c4f16a9bfde825de019f1f397e9
I20260926 07:44:42.277354 32652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc: Bootstrap starting.
I20260926 07:44:42.279346 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:42.273605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36793 (local address 127.0.0.1:41119)
0926 07:44:42.274453 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:42.274464 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:42.274512 (+ 48us) server_negotiation.cc:407] Connection header received
0926 07:44:42.276162 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:42.276183 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:42.276520 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:42.276824 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:42.277858 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:42.277865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:42.277878 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:42.277894 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:42.277946 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:42.277965 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:42.278122 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:42.278245 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 07:44:42.278459 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":386,"threads_started":1}
I20260926 07:44:42.280682 32652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:42.281880 32652 log.cc:824] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:42.283006 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:42.284538 32652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc: No bootstrap required, opened a new log
I20260926 07:44:42.289722 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:42.287399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:48813 (local address 127.0.0.1:41119)
0926 07:44:42.287563 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:42.287569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:42.287842 (+ 273us) server_negotiation.cc:407] Connection header received
0926 07:44:42.287959 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:42.287964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:42.288017 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:42.288124 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:42.288975 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:42.288983 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:42.288991 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:42.289009 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:42.289061 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:42.289080 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:42.289238 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:42.289389 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 07:44:42.289447 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:44:42.292969 32652 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260926 07:44:42.293340 32652 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:42.293416 32652 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4327ef310bc4b7aa8ab1c44287337dc, State: Initialized, Role: FOLLOWER
I20260926 07:44:42.293866 32652 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260926 07:44:42.294064 32652 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:42.294171 32652 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:42.294322 32652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:42.296556 32652 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260926 07:44:42.297017 32652 leader_election.cc:302] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4327ef310bc4b7aa8ab1c44287337dc; no voters:
I20260926 07:44:42.297587 32652 leader_election.cc:288] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:42.297740 32662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:42.298399 32662 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [term 1 LEADER]: Becoming Leader. State: Replica: e4327ef310bc4b7aa8ab1c44287337dc, State: Running, Role: LEADER
I20260926 07:44:42.298962 32662 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260926 07:44:42.299593 32652 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:42.301875 32663 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } } }
I20260926 07:44:42.302289 32663 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:42.302063 32664 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4327ef310bc4b7aa8ab1c44287337dc. Latest consensus state: current_term: 1 leader_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4327ef310bc4b7aa8ab1c44287337dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } } }
I20260926 07:44:42.302434 32664 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:42.303625 32669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:42.310341 32669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:42.319190 32669 catalog_manager.cc:1347] Generated new cluster ID: 727c62b79baf41cc801486b80b32ddaa
I20260926 07:44:42.319321 32669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:42.358525 32669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:42.358961 32669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:42.369201 32669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4327ef310bc4b7aa8ab1c44287337dc: Generated new TSK 0
I20260926 07:44:42.370105 32669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:42.383220 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32585
W20260926 07:44:42.384373 32647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:36793
W20260926 07:44:42.385056 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32585: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:42.385231 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32585
2026-09-26T07:44:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1885 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:42 dist-test-slave-x492 krb5kdc[32683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:42 dist-test-slave-x492 krb5kdc[32683](info): set up 2 sockets
Sep 26 07:44:42 dist-test-slave-x492 krb5kdc[32683](info): commencing operation
krb5kdc: starting...
W20260926 07:44:43.493798 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:43 dist-test-slave-x492 krb5kdc[32683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:43.744529 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:44.068578 32696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:44.068868 32696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:44.069025 32696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:44.076220 32696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:44.076328 32696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:44.076442 32696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:44.076552 32696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:44.076622 32696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:44.086308 32696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33779
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:44.087576 32696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:44.088838 32696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:44.100463 32704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:44.100585 32702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:44.100819 32701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:44.101469 32696 server_base.cc:1034] running on GCE node
I20260926 07:44:44.102214 32696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:44.103543 32696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:44.104746 32696 hybrid_clock.cc:648] HybridClock initialized: now 1790408684104696 us; error 38 us; skew 500 ppm
I20260926 07:44:44.105095 32696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:44 dist-test-slave-x492 krb5kdc[32683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:44.108170 32696 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:44.109659 32696 webserver.cc:460] Webserver started at http://127.0.0.1:38081/ using document root <none> and password file <none>
I20260926 07:44:44.110321 32696 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:44.110466 32696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:44.110759 32696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:44.113675 32696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "457043bb0e3248acafed027f6eedfaf0"
format_stamp: "Formatted at 2026-09-26 07:44:44 on dist-test-slave-x492"
server_key: "3fa8faba9568f553c3f2ac82e13c5ad3"
server_key_iv: "24695b046cccf7c60e66be65ed525e33"
server_key_version: "encryptionkey@0"
I20260926 07:44:44.114444 32696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "457043bb0e3248acafed027f6eedfaf0"
format_stamp: "Formatted at 2026-09-26 07:44:44 on dist-test-slave-x492"
server_key: "3fa8faba9568f553c3f2ac82e13c5ad3"
server_key_iv: "24695b046cccf7c60e66be65ed525e33"
server_key_version: "encryptionkey@0"
I20260926 07:44:44.119325 32696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:44.122603 32711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:44.123831 32696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:44:44.124002 32696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "457043bb0e3248acafed027f6eedfaf0"
format_stamp: "Formatted at 2026-09-26 07:44:44 on dist-test-slave-x492"
server_key: "3fa8faba9568f553c3f2ac82e13c5ad3"
server_key_iv: "24695b046cccf7c60e66be65ed525e33"
server_key_version: "encryptionkey@0"
I20260926 07:44:44.124183 32696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:44.141027 32696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:44.144811 32696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:44.145083 32696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:44.163481 32696 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33779
I20260926 07:44:44.163509 32762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33779 every 8 connection(s)
I20260926 07:44:44.164813 32696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:44.167625 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 32696
I20260926 07:44:44.167809 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:44.168241 11301 external_mini_cluster.cc:1428] Setting key 1582d090bf42df79e9d886a8cb1670f9
I20260926 07:44:44.168705 32763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:44.182575 32763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: Bootstrap starting.
Sep 26 07:44:44 dist-test-slave-x492 krb5kdc[32683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 07:44:44.186192 32763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:44.187302 32763 log.cc:824] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:44.190637 32763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: No bootstrap required, opened a new log
I20260926 07:44:44.191821 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:44.170052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:52943 (local address 127.0.0.1:33779)
0926 07:44:44.170799 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:44.170814 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:44.170861 (+ 47us) server_negotiation.cc:407] Connection header received
0926 07:44:44.173326 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:44.173354 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:44.173873 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:44.174298 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:44.175526 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:44.177553 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:44.178744 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:44.179826 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:44.180010 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:44.184322 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:44.184330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:44.184344 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:44.184390 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:44.187616 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:44.188552 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:44.188559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:44.188565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:44.188722 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:44.189271 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:44.189274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:44.189276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:44.189760 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:44.189995 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:44.190531 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 07:44:44.190846 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":308,"threads_started":1}
I20260926 07:44:44.195664 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:44.198619 32763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260926 07:44:44.199111 32763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:44.199239 32763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457043bb0e3248acafed027f6eedfaf0, State: Initialized, Role: FOLLOWER
I20260926 07:44:44.199950 32763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260926 07:44:44.200242 32763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:44.200410 32763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:44.200593 32763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:44.203689 32763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260926 07:44:44.204303 32763 leader_election.cc:302] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 457043bb0e3248acafed027f6eedfaf0; no voters:
I20260926 07:44:44.204936 32763 leader_election.cc:288] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:44.205057 305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:44.206707 305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [term 1 LEADER]: Becoming Leader. State: Replica: 457043bb0e3248acafed027f6eedfaf0, State: Running, Role: LEADER
I20260926 07:44:44.207346 32763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:44.207365 305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260926 07:44:44.218178 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:44.199949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:37913 (local address 127.0.0.1:33779)
0926 07:44:44.200128 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:44.200134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:44.200750 (+ 616us) server_negotiation.cc:407] Connection header received
0926 07:44:44.200952 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:44.200955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:44.201015 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:44.201085 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:44.202944 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:44.203663 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:44.205472 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:44.206175 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:44.206331 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:44.209692 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:44.209701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:44.209709 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:44.209760 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:44.214422 (+ 4662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:44.215423 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:44.215431 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:44.215435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:44.215605 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:44.216269 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:44.216273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:44.216276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:44.216664 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:44.217097 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:44.217445 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 07:44:44.217675 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:44:44.224480 307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457043bb0e3248acafed027f6eedfaf0. Latest consensus state: current_term: 1 leader_uuid: "457043bb0e3248acafed027f6eedfaf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } } }
I20260926 07:44:44.224601 306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "457043bb0e3248acafed027f6eedfaf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457043bb0e3248acafed027f6eedfaf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } } }
I20260926 07:44:44.225299 307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:44.225329 306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0 [sys.catalog]: This master's current role is: LEADER
W20260926 07:44:44.228996 321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:44:44.229100 321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:44:44.229243 322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:44.230755 322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:44.238490 322 catalog_manager.cc:1347] Generated new cluster ID: 0a69051b072346f790ace6e7dbceb27f
I20260926 07:44:44.238600 322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:44.257304 322 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:44.259028 322 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:44.268884 322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 457043bb0e3248acafed027f6eedfaf0: Generated new TSK 0
I20260926 07:44:44.269902 322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:44.281308 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 32696
I20260926 07:44:44.282500 32758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:44.375428 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 32696
W20260926 07:44:44.392321 32681 connection.cc:535] client connection to 127.0.0.1:33779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33779): Connection reset by peer (error 104)
2026-09-26T07:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1989 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:44 dist-test-slave-x492 krb5kdc[328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:44 dist-test-slave-x492 krb5kdc[328](info): set up 2 sockets
Sep 26 07:44:44 dist-test-slave-x492 krb5kdc[328](info): commencing operation
krb5kdc: starting...
W20260926 07:44:45.473634 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:45 dist-test-slave-x492 krb5kdc[328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:45.724594 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:46.053963 341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:46.054304 341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:46.054492 341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:46.061574 341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:46.061697 341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:46.061790 341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:46.061903 341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:46.061988 341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:46.071529 341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:46.072880 341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:46.074241 341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:46.085635 346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:46.085731 347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:46.085682 349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:46.086651 341 server_base.cc:1034] running on GCE node
I20260926 07:44:46.087438 341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:46.088722 341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:46.089949 341 hybrid_clock.cc:648] HybridClock initialized: now 1790408686089899 us; error 60 us; skew 500 ppm
I20260926 07:44:46.090293 341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:46 dist-test-slave-x492 krb5kdc[328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408686, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:46.093425 341 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:46.094817 341 webserver.cc:460] Webserver started at http://127.0.0.1:36737/ using document root <none> and password file <none>
I20260926 07:44:46.095489 341 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:46.095630 341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:46.095928 341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:46.098852 341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "7975750de68f4795830e76e924064ae5"
format_stamp: "Formatted at 2026-09-26 07:44:46 on dist-test-slave-x492"
server_key: "49c939e8e83e5ea9d19fb047b277367c"
server_key_iv: "5e4251eb51bc6bb9594d76f89f498515"
server_key_version: "encryptionkey@0"
I20260926 07:44:46.099584 341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "7975750de68f4795830e76e924064ae5"
format_stamp: "Formatted at 2026-09-26 07:44:46 on dist-test-slave-x492"
server_key: "49c939e8e83e5ea9d19fb047b277367c"
server_key_iv: "5e4251eb51bc6bb9594d76f89f498515"
server_key_version: "encryptionkey@0"
I20260926 07:44:46.104480 341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:44:46.107882 356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:46.109273 341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:46.109448 341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "7975750de68f4795830e76e924064ae5"
format_stamp: "Formatted at 2026-09-26 07:44:46 on dist-test-slave-x492"
server_key: "49c939e8e83e5ea9d19fb047b277367c"
server_key_iv: "5e4251eb51bc6bb9594d76f89f498515"
server_key_version: "encryptionkey@0"
I20260926 07:44:46.109638 341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:46.134450 341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:46.138839 341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:46.139122 341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:46.157263 341 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37483
I20260926 07:44:46.157315 407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37483 every 8 connection(s)
I20260926 07:44:46.158692 341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:46.162292 408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:46.168256 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 341
I20260926 07:44:46.168452 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:46.168874 11301 external_mini_cluster.cc:1428] Setting key 63e313c2c2147483fbb59a6d985d1c56
I20260926 07:44:46.172587 408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5: Bootstrap starting.
I20260926 07:44:46.176066 408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:46.177274 408 log.cc:824] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:46.179970 408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5: No bootstrap required, opened a new log
I20260926 07:44:46.181842 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:46.170597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:52541 (local address 127.0.0.1:37483)
0926 07:44:46.171330 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:46.171341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:46.171379 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:44:46.173010 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:46.173030 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:46.173465 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:46.173787 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:46.174402 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:46.175626 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:46.177536 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:46.178212 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:46.178421 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:46.179600 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:46.179608 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:46.179621 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:46.180172 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:46.180244 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:46.180270 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:46.180449 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:46.180713 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 07:44:46.180867 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":336,"threads_started":1}
I20260926 07:44:46.185467 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:46.187827 408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260926 07:44:46.188210 408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:46.188364 408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7975750de68f4795830e76e924064ae5, State: Initialized, Role: FOLLOWER
I20260926 07:44:46.189026 408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260926 07:44:46.189309 408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:46.189456 408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:46.189646 408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:46.192689 408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260926 07:44:46.193284 408 leader_election.cc:302] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7975750de68f4795830e76e924064ae5; no voters:
I20260926 07:44:46.193929 408 leader_election.cc:288] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:46.194039 418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:46.196205 408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:46.195873 418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [term 1 LEADER]: Becoming Leader. State: Replica: 7975750de68f4795830e76e924064ae5, State: Running, Role: LEADER
I20260926 07:44:46.196812 418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } }
I20260926 07:44:46.197561 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:46.189941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:40785 (local address 127.0.0.1:37483)
0926 07:44:46.190144 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:46.190152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:46.190655 (+ 503us) server_negotiation.cc:407] Connection header received
0926 07:44:46.190846 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:46.190849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:46.190913 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:46.190996 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:46.191566 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:46.192291 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:46.194260 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:46.195194 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:46.195375 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:46.196593 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:46.196602 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:46.196608 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:46.196632 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:46.196684 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:46.196705 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:46.196840 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:46.197153 (+ 313us) server_negotiation.cc:299] Negotiation successful
0926 07:44:46.197228 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":61}
I20260926 07:44:46.199818 419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7975750de68f4795830e76e924064ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } } }
I20260926 07:44:46.199831 420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7975750de68f4795830e76e924064ae5. Latest consensus state: current_term: 1 leader_uuid: "7975750de68f4795830e76e924064ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975750de68f4795830e76e924064ae5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37483 } } }
I20260926 07:44:46.200230 420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:46.200230 419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:46.206264 423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:46.211470 423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:46.219951 423 catalog_manager.cc:1347] Generated new cluster ID: 115c9e8a20be470cb9a43f953d918a61
I20260926 07:44:46.220109 423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:46.230394 423 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:46.231703 423 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:46.242980 423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7975750de68f4795830e76e924064ae5: Generated new TSK 0
I20260926 07:44:46.244030 423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:46.256400 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 341
W20260926 07:44:46.257506 403 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:52541
W20260926 07:44:46.258253 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 341: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:46.258394 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 341
W20260926 07:44:46.275229 326 connection.cc:535] client connection to 127.0.0.1:37483 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37483): Connection reset by peer (error 104)
2026-09-26T07:44:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1882 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:46 dist-test-slave-x492 krb5kdc[439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:46 dist-test-slave-x492 krb5kdc[439](info): set up 2 sockets
Sep 26 07:44:46 dist-test-slave-x492 krb5kdc[439](info): commencing operation
krb5kdc: starting...
W20260926 07:44:47.362955 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:47 dist-test-slave-x492 krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 07:44:47.604669 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:47.930056 452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:47.930338 452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:47.930529 452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:47.937832 452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:47.937942 452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:47.938036 452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:47.938138 452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:47.938226 452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:47.948160 452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:47.949652 452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:47.950982 452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:47.963066 460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:47.963047 458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:47.963647 452 server_base.cc:1034] running on GCE node
W20260926 07:44:47.963186 457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:47.964476 452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:47.965962 452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:47.967211 452 hybrid_clock.cc:648] HybridClock initialized: now 1790408687967161 us; error 58 us; skew 500 ppm
I20260926 07:44:47.967573 452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:47 dist-test-slave-x492 krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:47.970700 452 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 07:44:47.972155 452 webserver.cc:460] Webserver started at http://127.0.0.1:42935/ using document root <none> and password file <none>
I20260926 07:44:47.972848 452 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:47.972993 452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:47.973361 452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:47.977490 452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "89cc4435d2a9457999fd1d6296a2347e"
format_stamp: "Formatted at 2026-09-26 07:44:47 on dist-test-slave-x492"
server_key: "1d9b7e6200bba780f369f75d9f4ef947"
server_key_iv: "6e609de686eb0942d5f750f95ff74592"
server_key_version: "encryptionkey@0"
I20260926 07:44:47.978542 452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "89cc4435d2a9457999fd1d6296a2347e"
format_stamp: "Formatted at 2026-09-26 07:44:47 on dist-test-slave-x492"
server_key: "1d9b7e6200bba780f369f75d9f4ef947"
server_key_iv: "6e609de686eb0942d5f750f95ff74592"
server_key_version: "encryptionkey@0"
I20260926 07:44:47.985234 452 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 07:44:47.988569 467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:47.989933 452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:47.990119 452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "89cc4435d2a9457999fd1d6296a2347e"
format_stamp: "Formatted at 2026-09-26 07:44:47 on dist-test-slave-x492"
server_key: "1d9b7e6200bba780f369f75d9f4ef947"
server_key_iv: "6e609de686eb0942d5f750f95ff74592"
server_key_version: "encryptionkey@0"
I20260926 07:44:47.990314 452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:47.995947 452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:47.999995 452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:48.000283 452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:48.018596 452 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36923
I20260926 07:44:48.018615 518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36923 every 8 connection(s)
I20260926 07:44:48.020046 452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:48.023721 519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:48.027500 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 452
I20260926 07:44:48.027665 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:48.028183 11301 external_mini_cluster.cc:1428] Setting key 37b154482a918daad943dd77b564d36d
I20260926 07:44:48.036257 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:48.029957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:55249 (local address 127.0.0.1:36923)
0926 07:44:48.030702 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:48.030713 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:48.030754 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:44:48.032363 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:48.032385 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:48.032736 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:48.033055 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:48.034478 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:48.034486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:48.034503 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:48.034522 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:48.034612 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:48.034641 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:48.034873 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:48.034996 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 07:44:48.035208 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":329,"threads_started":1}
I20260926 07:44:48.037089 519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e: Bootstrap starting.
I20260926 07:44:48.040062 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:48.040562 519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:48.041594 519 log.cc:824] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:48.044557 519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e: No bootstrap required, opened a new log
I20260926 07:44:48.047010 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:48.044493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:40001 (local address 127.0.0.1:36923)
0926 07:44:48.044666 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:48.044674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:48.045033 (+ 359us) server_negotiation.cc:407] Connection header received
0926 07:44:48.045324 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:48.045328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:48.045371 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:48.045477 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:48.046367 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:48.046372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:48.046377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:48.046394 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:48.046439 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:48.046454 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:48.046581 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:48.046692 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 07:44:48.046754 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 07:44:48.052305 519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } }
I20260926 07:44:48.052682 519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:48.052796 519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89cc4435d2a9457999fd1d6296a2347e, State: Initialized, Role: FOLLOWER
I20260926 07:44:48.053478 519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } }
I20260926 07:44:48.053782 519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:48.053939 519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:48.054129 519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:48.056406 519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } }
I20260926 07:44:48.056972 519 leader_election.cc:302] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89cc4435d2a9457999fd1d6296a2347e; no voters:
I20260926 07:44:48.057706 519 leader_election.cc:288] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:48.057832 529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:48.058555 529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [term 1 LEADER]: Becoming Leader. State: Replica: 89cc4435d2a9457999fd1d6296a2347e, State: Running, Role: LEADER
I20260926 07:44:48.059162 529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } }
I20260926 07:44:48.059962 519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:48.062031 531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89cc4435d2a9457999fd1d6296a2347e. Latest consensus state: current_term: 1 leader_uuid: "89cc4435d2a9457999fd1d6296a2347e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } } }
I20260926 07:44:48.062490 531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:48.062458 530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89cc4435d2a9457999fd1d6296a2347e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cc4435d2a9457999fd1d6296a2347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36923 } } }
I20260926 07:44:48.062747 530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:48.064337 537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:48.070621 537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:48.079340 537 catalog_manager.cc:1347] Generated new cluster ID: f7c0c89f05ff4cff8f2d6e87b1c98973
I20260926 07:44:48.079476 537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:48.102169 537 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:48.102705 537 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:48.114379 537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89cc4435d2a9457999fd1d6296a2347e: Generated new TSK 0
I20260926 07:44:48.115337 537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:48.129557 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 452
W20260926 07:44:48.131091 514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:55249
W20260926 07:44:48.131846 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 452: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:48.131994 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 452
2026-09-26T07:44:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1870 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:48 dist-test-slave-x492 krb5kdc[550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:48 dist-test-slave-x492 krb5kdc[550](info): set up 2 sockets
Sep 26 07:44:48 dist-test-slave-x492 krb5kdc[550](info): commencing operation
krb5kdc: starting...
W20260926 07:44:49.221699 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:49 dist-test-slave-x492 krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:49.456662 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37067
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:49.774322 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:49.774642 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:49.774813 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:49.781738 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:49.781884 563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:49.781999 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:49.782111 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:49.782200 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:49.791637 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:37067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:49.792927 563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:49.794275 563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:49.805891 569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:49.806152 568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:49.806200 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:49.806816 563 server_base.cc:1034] running on GCE node
I20260926 07:44:49.807507 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:49.808781 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:49.810012 563 hybrid_clock.cc:648] HybridClock initialized: now 1790408689809972 us; error 50 us; skew 500 ppm
I20260926 07:44:49.810356 563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:49 dist-test-slave-x492 krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408689, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:49.813423 563 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:49.814906 563 webserver.cc:460] Webserver started at http://127.11.9.126:45347/ using document root <none> and password file <none>
I20260926 07:44:49.815565 563 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:49.815711 563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:49.816009 563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:49.818913 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "131e562f7ce0497b9842821582c97d8f"
format_stamp: "Formatted at 2026-09-26 07:44:49 on dist-test-slave-x492"
server_key: "5dd9f0cce20e4fea51313ab7d11e3e9f"
server_key_iv: "332a9d586757533178981ea18ac79cd0"
server_key_version: "encryptionkey@0"
I20260926 07:44:49.819674 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "131e562f7ce0497b9842821582c97d8f"
format_stamp: "Formatted at 2026-09-26 07:44:49 on dist-test-slave-x492"
server_key: "5dd9f0cce20e4fea51313ab7d11e3e9f"
server_key_iv: "332a9d586757533178981ea18ac79cd0"
server_key_version: "encryptionkey@0"
I20260926 07:44:49.824612 563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:44:49.828070 578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:49.829406 563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:49.829581 563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "131e562f7ce0497b9842821582c97d8f"
format_stamp: "Formatted at 2026-09-26 07:44:49 on dist-test-slave-x492"
server_key: "5dd9f0cce20e4fea51313ab7d11e3e9f"
server_key_iv: "332a9d586757533178981ea18ac79cd0"
server_key_version: "encryptionkey@0"
I20260926 07:44:49.829769 563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:49.853288 563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:49.857115 563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:49.857407 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:49.875385 563 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:37067
I20260926 07:44:49.875417 629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:37067 every 8 connection(s)
I20260926 07:44:49.876752 563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:49.878100 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 563
I20260926 07:44:49.878266 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:49.878813 11301 external_mini_cluster.cc:1428] Setting key 77f3dae6c82465c07b1b109dfb3414b5
I20260926 07:44:49.881052 630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 07:44:49 dist-test-slave-x492 krb5kdc[550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:49.893810 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f: Bootstrap starting.
I20260926 07:44:49.897928 630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:49.899109 630 log.cc:824] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:49.902328 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:49.880534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48723 (local address 127.11.9.126:37067)
0926 07:44:49.881355 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:49.881367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:49.881407 (+ 40us) server_negotiation.cc:407] Connection header received
0926 07:44:49.882998 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:49.883019 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:49.883389 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:49.883722 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:49.885442 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:49.886777 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:49.888465 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:49.889112 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:49.889309 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:49.893896 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:49.893906 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:49.893924 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:49.893999 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:49.897815 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:49.898546 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:49.898554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:49.898561 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:49.898768 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:49.899222 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:49.899229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:49.899233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:49.899928 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:49.900182 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:49.900777 (+ 595us) server_negotiation.cc:299] Negotiation successful
0926 07:44:49.901083 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":362,"threads_started":1}
I20260926 07:44:49.902424 630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f: No bootstrap required, opened a new log
I20260926 07:44:49.906085 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:49.910549 630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } }
I20260926 07:44:49.910904 630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:49.911026 630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131e562f7ce0497b9842821582c97d8f, State: Initialized, Role: FOLLOWER
I20260926 07:44:49.911599 630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } }
I20260926 07:44:49.911820 630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:49.911971 630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:49.912122 630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:49.914872 630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } }
I20260926 07:44:49.915401 630 leader_election.cc:302] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 131e562f7ce0497b9842821582c97d8f; no voters:
I20260926 07:44:49.916067 630 leader_election.cc:288] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:49.916204 640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:49.917372 640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [term 1 LEADER]: Becoming Leader. State: Replica: 131e562f7ce0497b9842821582c97d8f, State: Running, Role: LEADER
I20260926 07:44:49.918044 640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } }
I20260926 07:44:49.918219 630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:49.922363 641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "131e562f7ce0497b9842821582c97d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } } }
I20260926 07:44:49.922832 641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:49.927376 642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 131e562f7ce0497b9842821582c97d8f. Latest consensus state: current_term: 1 leader_uuid: "131e562f7ce0497b9842821582c97d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131e562f7ce0497b9842821582c97d8f" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 37067 } } }
I20260926 07:44:49.928284 642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:49.929366 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:49.910237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36259 (local address 127.11.9.126:37067)
0926 07:44:49.910418 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:49.910424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:49.910907 (+ 483us) server_negotiation.cc:407] Connection header received
0926 07:44:49.911076 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:49.911082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:49.911158 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:49.911256 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:49.912849 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:49.914077 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:49.915293 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:49.916043 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:49.916185 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:49.919350 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:49.919359 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:49.919367 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:49.919411 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:49.923977 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:49.925254 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:49.925262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:49.925268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:49.925472 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:49.926197 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:49.926204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:49.926210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:49.926710 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:49.928385 (+ 1675us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:49.928614 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 07:44:49.928842 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 07:44:49.933326 648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:49.938948 648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:49.947641 648 catalog_manager.cc:1347] Generated new cluster ID: 6df64916d29540b880777043d5ce093d
I20260926 07:44:49.947765 648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:49.958832 648 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:49.960621 648 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:49.974450 648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 131e562f7ce0497b9842821582c97d8f: Generated new TSK 0
I20260926 07:44:49.975469 648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:49.988062 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 563
I20260926 07:44:49.989379 625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:50.080197 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 563
W20260926 07:44:50.097043 548 connection.cc:535] client connection to 127.11.9.126:37067 recv error: Network error: failed to read from TLS socket (remote: 127.11.9.126:37067): Connection reset by peer (error 104)
2026-09-26T07:44:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1950 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:50 dist-test-slave-x492 krb5kdc[662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:50 dist-test-slave-x492 krb5kdc[662](info): set up 2 sockets
Sep 26 07:44:50 dist-test-slave-x492 krb5kdc[662](info): commencing operation
krb5kdc: starting...
W20260926 07:44:51.177671 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:51 dist-test-slave-x492 krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:51.416684 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34021
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:42195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:34021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:51.736104 675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:51.736366 675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:51.736549 675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:51.743561 675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:51.743652 675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:51.743696 675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:51.743746 675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:51.743808 675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:51.753042 675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:42195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:34021
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:34021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:51.754271 675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:51.755530 675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:51.766901 680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:51.766954 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:51.767200 681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:51.767858 675 server_base.cc:1034] running on GCE node
I20260926 07:44:51.768569 675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:51.769994 675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:51.771224 675 hybrid_clock.cc:648] HybridClock initialized: now 1790408691771158 us; error 27 us; skew 500 ppm
I20260926 07:44:51.771837 675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:51 dist-test-slave-x492 krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408691, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:51.774976 675 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:51.776350 675 webserver.cc:460] Webserver started at http://127.11.9.126:34405/ using document root <none> and password file <none>
I20260926 07:44:51.776924 675 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:51.777027 675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:51.777328 675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:51.780153 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "14ee51ffe0a14b30879a641ee5972a59"
format_stamp: "Formatted at 2026-09-26 07:44:51 on dist-test-slave-x492"
server_key: "9b8a1b2dffe6a2e68c0a588c24c95021"
server_key_iv: "9d70b72bfe265f2e84d233a5ef646994"
server_key_version: "encryptionkey@0"
I20260926 07:44:51.780830 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "14ee51ffe0a14b30879a641ee5972a59"
format_stamp: "Formatted at 2026-09-26 07:44:51 on dist-test-slave-x492"
server_key: "9b8a1b2dffe6a2e68c0a588c24c95021"
server_key_iv: "9d70b72bfe265f2e84d233a5ef646994"
server_key_version: "encryptionkey@0"
I20260926 07:44:51.785780 675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 07:44:51.788900 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:51.790158 675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:44:51.790283 675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "14ee51ffe0a14b30879a641ee5972a59"
format_stamp: "Formatted at 2026-09-26 07:44:51 on dist-test-slave-x492"
server_key: "9b8a1b2dffe6a2e68c0a588c24c95021"
server_key_iv: "9d70b72bfe265f2e84d233a5ef646994"
server_key_version: "encryptionkey@0"
I20260926 07:44:51.790434 675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:51.807890 675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:51.811627 675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:51.811849 675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:51.829668 675 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:34021
I20260926 07:44:51.829716 741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:34021 every 8 connection(s)
I20260926 07:44:51.831015 675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:51.834563 742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:51.839459 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 675
I20260926 07:44:51.839663 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:51.840196 11301 external_mini_cluster.cc:1428] Setting key b1a03107d5cc88cca62072a60ee37a0b
I20260926 07:44:51.845640 742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59: Bootstrap starting.
I20260926 07:44:51.848541 742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:51.849608 742 log.cc:824] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:51.852877 742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59: No bootstrap required, opened a new log
I20260926 07:44:51.853477 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:51.841949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56303 (local address 127.11.9.126:34021)
0926 07:44:51.842691 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:51.842710 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:51.842752 (+ 42us) server_negotiation.cc:407] Connection header received
0926 07:44:51.844451 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:51.844471 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:51.844836 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:51.845237 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:51.845787 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:51.847108 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:51.848373 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:51.849118 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:51.849408 (+ 290us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:51.850799 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:51.850807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:51.850819 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:51.851507 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:51.851606 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:51.851639 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:51.851810 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:51.852052 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 07:44:51.852213 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":308,"threads_started":1}
I20260926 07:44:51.857383 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:51.863314 742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } }
I20260926 07:44:51.863700 742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:51.863857 742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee51ffe0a14b30879a641ee5972a59, State: Initialized, Role: FOLLOWER
I20260926 07:44:51.864459 742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } }
I20260926 07:44:51.864724 742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:51.864876 742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:51.865052 742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:51.867249 742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } }
I20260926 07:44:51.867769 742 leader_election.cc:302] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14ee51ffe0a14b30879a641ee5972a59; no voters:
I20260926 07:44:51.868427 742 leader_election.cc:288] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:51.868530 752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:51.869004 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:51.862462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37207 (local address 127.11.9.126:34021)
0926 07:44:51.862631 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:51.862637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:51.862962 (+ 325us) server_negotiation.cc:407] Connection header received
0926 07:44:51.863113 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:51.863121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:51.863189 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:51.863288 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:51.863770 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:51.864618 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:51.866060 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:51.866538 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:51.866678 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:51.867862 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:51.867867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:51.867871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:51.867887 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:51.867987 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:51.868008 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:51.868138 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:51.868328 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 07:44:51.868387 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:44:51.871187 742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:51.876428 752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [term 1 LEADER]: Becoming Leader. State: Replica: 14ee51ffe0a14b30879a641ee5972a59, State: Running, Role: LEADER
I20260926 07:44:51.877211 752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } }
I20260926 07:44:51.880829 754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14ee51ffe0a14b30879a641ee5972a59. Latest consensus state: current_term: 1 leader_uuid: "14ee51ffe0a14b30879a641ee5972a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } } }
I20260926 07:44:51.881294 754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:51.880816 753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14ee51ffe0a14b30879a641ee5972a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ee51ffe0a14b30879a641ee5972a59" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 34021 } } }
I20260926 07:44:51.881814 753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:51.886687 764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:51.891139 764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:51.899225 764 catalog_manager.cc:1347] Generated new cluster ID: 33b94f6ed98b47bfa65133b95f419095
I20260926 07:44:51.899348 764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:51.918367 764 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:51.919795 764 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:51.941023 764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14ee51ffe0a14b30879a641ee5972a59: Generated new TSK 0
I20260926 07:44:51.941905 764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:51.960707 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 675
W20260926 07:44:51.961817 737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56303
W20260926 07:44:51.962561 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 675: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:51.962697 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 675
W20260926 07:44:51.984120 660 connection.cc:535] client connection to 127.11.9.126:34021 recv error: Network error: failed to read from TLS socket (remote: 127.11.9.126:34021): Connection reset by peer (error 104)
2026-09-26T07:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:52 dist-test-slave-x492 krb5kdc[773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:52 dist-test-slave-x492 krb5kdc[773](info): set up 2 sockets
Sep 26 07:44:52 dist-test-slave-x492 krb5kdc[773](info): commencing operation
krb5kdc: starting...
W20260926 07:44:53.069909 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:53 dist-test-slave-x492 krb5kdc[773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:53.324457 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42599
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:53.648166 786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:53.648450 786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:53.648540 786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:53.655923 786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:53.656038 786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:53.656085 786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:53.656147 786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:53.656190 786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:53.666079 786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:42599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:53.667356 786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:53.668642 786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:53.680419 794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:53.680474 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:53.680670 791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:53.681464 786 server_base.cc:1034] running on GCE node
I20260926 07:44:53.682178 786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:53.683540 786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:53.684804 786 hybrid_clock.cc:648] HybridClock initialized: now 1790408693684767 us; error 48 us; skew 500 ppm
I20260926 07:44:53.685231 786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:53 dist-test-slave-x492 krb5kdc[773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:53.694120 786 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:53.695644 786 webserver.cc:460] Webserver started at http://127.11.9.126:44041/ using document root <none> and password file <none>
I20260926 07:44:53.696313 786 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:53.696460 786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:53.696758 786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:53.699704 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "8794fd7fba914171868f081bb3446b22"
format_stamp: "Formatted at 2026-09-26 07:44:53 on dist-test-slave-x492"
server_key: "1c93efa9b6c6224f398d85eb2fccfef2"
server_key_iv: "6c080549d82823e25bbb90d960d6eb02"
server_key_version: "encryptionkey@0"
I20260926 07:44:53.700443 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "8794fd7fba914171868f081bb3446b22"
format_stamp: "Formatted at 2026-09-26 07:44:53 on dist-test-slave-x492"
server_key: "1c93efa9b6c6224f398d85eb2fccfef2"
server_key_iv: "6c080549d82823e25bbb90d960d6eb02"
server_key_version: "encryptionkey@0"
I20260926 07:44:53.705703 786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 07:44:53.709122 801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:53.710608 786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 07:44:53.710790 786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "8794fd7fba914171868f081bb3446b22"
format_stamp: "Formatted at 2026-09-26 07:44:53 on dist-test-slave-x492"
server_key: "1c93efa9b6c6224f398d85eb2fccfef2"
server_key_iv: "6c080549d82823e25bbb90d960d6eb02"
server_key_version: "encryptionkey@0"
I20260926 07:44:53.711012 786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:53.716656 786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:53.721091 786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:53.721424 786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:53.739719 786 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:42599
I20260926 07:44:53.739764 852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:42599 every 8 connection(s)
I20260926 07:44:53.741129 786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:53.744726 853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:53.747215 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 786
I20260926 07:44:53.747380 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:53.747929 11301 external_mini_cluster.cc:1428] Setting key 36b9c5839cec086513a7afc105e6d4d8
I20260926 07:44:53.755538 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:53.749735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57569 (local address 127.11.9.126:42599)
0926 07:44:53.750515 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:53.750525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:53.750566 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:44:53.752196 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:53.752217 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:53.752537 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:53.752838 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:53.753942 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:53.753948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:53.753961 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:53.753987 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:53.754049 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:53.754072 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:53.754270 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:53.754377 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 07:44:53.754620 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":370,"threads_started":1}
I20260926 07:44:53.756865 853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22: Bootstrap starting.
I20260926 07:44:53.759262 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:53.760102 853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:53.761317 853 log.cc:824] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:53.764539 853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22: No bootstrap required, opened a new log
I20260926 07:44:53.766736 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:53.764246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40219 (local address 127.11.9.126:42599)
0926 07:44:53.764417 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:53.764422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:53.764879 (+ 457us) server_negotiation.cc:407] Connection header received
0926 07:44:53.764996 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:53.765002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:53.765061 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:53.765187 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:53.766085 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:53.766091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:53.766095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:53.766112 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:53.766159 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:53.766183 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:53.766318 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:53.766392 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 07:44:53.766463 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:44:53.773414 853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } }
I20260926 07:44:53.773742 853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:53.773813 853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8794fd7fba914171868f081bb3446b22, State: Initialized, Role: FOLLOWER
I20260926 07:44:53.774289 853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } }
I20260926 07:44:53.774485 853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:53.774580 853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:53.774688 853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:53.776749 853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } }
I20260926 07:44:53.777206 853 leader_election.cc:302] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8794fd7fba914171868f081bb3446b22; no voters:
I20260926 07:44:53.777927 853 leader_election.cc:288] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:53.778003 863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:53.778710 863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [term 1 LEADER]: Becoming Leader. State: Replica: 8794fd7fba914171868f081bb3446b22, State: Running, Role: LEADER
I20260926 07:44:53.779243 863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } }
I20260926 07:44:53.779971 853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:53.782037 865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8794fd7fba914171868f081bb3446b22. Latest consensus state: current_term: 1 leader_uuid: "8794fd7fba914171868f081bb3446b22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } } }
I20260926 07:44:53.782130 864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8794fd7fba914171868f081bb3446b22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8794fd7fba914171868f081bb3446b22" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 42599 } } }
I20260926 07:44:53.782416 865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:53.782522 864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:53.784090 869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:53.789214 869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:53.797670 869 catalog_manager.cc:1347] Generated new cluster ID: afdb86eabb454ee29d9e25772febd683
I20260926 07:44:53.797776 869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:53.849251 869 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:53.849676 869 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:53.855607 869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8794fd7fba914171868f081bb3446b22: Generated new TSK 0
I20260926 07:44:53.856360 869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:53.870083 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 786
W20260926 07:44:53.871114 848 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57569
W20260926 07:44:53.871775 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 786: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:53.871939 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 786
2026-09-26T07:44:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1903 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:53 dist-test-slave-x492 krb5kdc[884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:53 dist-test-slave-x492 krb5kdc[884](info): set up 2 sockets
Sep 26 07:44:53 dist-test-slave-x492 krb5kdc[884](info): commencing operation
krb5kdc: starting...
W20260926 07:44:54.973613 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:55.212266 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35415
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:55.532613 897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:55.532958 897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:55.533123 897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:55.540220 897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:55.540395 897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:55.540524 897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:55.540663 897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:55.540745 897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:55.550403 897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:55.551702 897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:55.553004 897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:55.564849 902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:55.564946 905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:55.564955 903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:55.565806 897 server_base.cc:1034] running on GCE node
I20260926 07:44:55.566576 897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:55.567993 897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:55.569252 897 hybrid_clock.cc:648] HybridClock initialized: now 1790408695569217 us; error 50 us; skew 500 ppm
I20260926 07:44:55.569597 897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408695, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:55.572659 897 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:55.574129 897 webserver.cc:460] Webserver started at http://127.11.9.126:37973/ using document root <none> and password file <none>
I20260926 07:44:55.574736 897 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:55.574829 897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:55.575057 897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:55.578074 897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "4e10251ea3f84bffb8782a8a0ee41deb"
format_stamp: "Formatted at 2026-09-26 07:44:55 on dist-test-slave-x492"
server_key: "e6c9eabbc48640da6681aba9dbe37aa2"
server_key_iv: "867559122c9a6954db4ae53dfd82267c"
server_key_version: "encryptionkey@0"
I20260926 07:44:55.578758 897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "4e10251ea3f84bffb8782a8a0ee41deb"
format_stamp: "Formatted at 2026-09-26 07:44:55 on dist-test-slave-x492"
server_key: "e6c9eabbc48640da6681aba9dbe37aa2"
server_key_iv: "867559122c9a6954db4ae53dfd82267c"
server_key_version: "encryptionkey@0"
I20260926 07:44:55.583676 897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:55.587009 912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:55.588248 897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:44:55.588522 897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "4e10251ea3f84bffb8782a8a0ee41deb"
format_stamp: "Formatted at 2026-09-26 07:44:55 on dist-test-slave-x492"
server_key: "e6c9eabbc48640da6681aba9dbe37aa2"
server_key_iv: "867559122c9a6954db4ae53dfd82267c"
server_key_version: "encryptionkey@0"
I20260926 07:44:55.588683 897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:55.609021 897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:55.612929 897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:55.613255 897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:55.631579 897 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:35415
I20260926 07:44:55.631620 963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:35415 every 8 connection(s)
I20260926 07:44:55.632920 897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:55.634539 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 897
I20260926 07:44:55.634712 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:55.635244 11301 external_mini_cluster.cc:1428] Setting key cce3c091eeac6af04cab8183f1c95088
I20260926 07:44:55.637516 964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:44:55.650206 964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb: Bootstrap starting.
I20260926 07:44:55.654259 964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:55.655474 964 log.cc:824] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:55.658465 964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb: No bootstrap required, opened a new log
I20260926 07:44:55.658816 967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:55.636955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41139 (local address 127.11.9.126:35415)
0926 07:44:55.637738 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:55.637752 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:55.637806 (+ 54us) server_negotiation.cc:407] Connection header received
0926 07:44:55.640145 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:55.640170 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:55.640676 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:55.641108 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:55.642427 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:55.644451 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:55.645791 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:55.646791 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:55.646986 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:55.650879 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:55.650898 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:55.650915 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:55.650961 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:55.654940 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:55.655581 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:55.655589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:55.655598 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:55.655790 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:55.656158 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:55.656165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:55.656173 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:55.656719 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:55.657025 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:55.657464 (+ 439us) server_negotiation.cc:299] Negotiation successful
0926 07:44:55.657860 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":344,"threads_started":1}
I20260926 07:44:55.662456 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:55.666316 964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } }
I20260926 07:44:55.666728 964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:55.666860 964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e10251ea3f84bffb8782a8a0ee41deb, State: Initialized, Role: FOLLOWER
I20260926 07:44:55.667450 964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } }
I20260926 07:44:55.667687 964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:55.667829 964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:55.668010 964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:55.670218 964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } }
I20260926 07:44:55.670686 964 leader_election.cc:302] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e10251ea3f84bffb8782a8a0ee41deb; no voters:
I20260926 07:44:55.671357 964 leader_election.cc:288] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:55.671486 974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:55.673832 964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:55.678489 974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [term 1 LEADER]: Becoming Leader. State: Replica: 4e10251ea3f84bffb8782a8a0ee41deb, State: Running, Role: LEADER
I20260926 07:44:55.679266 974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } }
I20260926 07:44:55.680614 967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:55.666549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40319 (local address 127.11.9.126:35415)
0926 07:44:55.666734 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:55.666741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:55.667203 (+ 462us) server_negotiation.cc:407] Connection header received
0926 07:44:55.667375 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:55.667380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:55.667448 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:55.667545 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:55.669007 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:55.669806 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:55.671124 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:55.671726 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:55.671889 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:55.674662 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:55.674668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:55.674673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:44:55.674712 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:55.677857 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:55.678499 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:55.678504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:55.678509 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:55.678712 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:44:55.679109 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:44:55.679114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:44:55.679118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:44:55.679446 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:55.679734 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:55.680059 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 07:44:55.680257 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:44:55.682578 975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } } }
I20260926 07:44:55.683045 975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:55.687577 976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e10251ea3f84bffb8782a8a0ee41deb. Latest consensus state: current_term: 1 leader_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e10251ea3f84bffb8782a8a0ee41deb" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35415 } } }
I20260926 07:44:55.688025 976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:55.689066 983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:55.694767 983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:55.703759 983 catalog_manager.cc:1347] Generated new cluster ID: 0e191bb6a4374fca960a09310c3c2896
I20260926 07:44:55.703903 983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:55.710713 983 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:55.712033 983 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:55.718189 983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e10251ea3f84bffb8782a8a0ee41deb: Generated new TSK 0
I20260926 07:44:55.719027 983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:55.724433 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 897
I20260926 07:44:55.726235 959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:44:55.813550 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 897
2026-09-26T07:44:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[997](info): set up 2 sockets
Sep 26 07:44:55 dist-test-slave-x492 krb5kdc[997](info): commencing operation
krb5kdc: starting...
W20260926 07:44:56.917763 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:57 dist-test-slave-x492 krb5kdc[997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:57.168378 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:41945
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:57.504963 1010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:57.505259 1010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:57.505353 1010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:57.512375 1010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:57.512482 1010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:57.512532 1010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:57.512594 1010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:57.512638 1010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:57.522401 1010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:57.523614 1010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:57.524923 1010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:57.536684 1018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:57.536895 1015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:57.536779 1016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:57.538014 1010 server_base.cc:1034] running on GCE node
I20260926 07:44:57.538738 1010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:57.540078 1010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:57.541270 1010 hybrid_clock.cc:648] HybridClock initialized: now 1790408697541210 us; error 41 us; skew 500 ppm
I20260926 07:44:57.541632 1010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:57 dist-test-slave-x492 krb5kdc[997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408697, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:57.544783 1010 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:57.546213 1010 webserver.cc:460] Webserver started at http://127.11.9.126:45313/ using document root <none> and password file <none>
I20260926 07:44:57.546875 1010 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:57.546980 1010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:57.547245 1010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:57.550436 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03"
format_stamp: "Formatted at 2026-09-26 07:44:57 on dist-test-slave-x492"
server_key: "12eebeb98588a22435c3a464e448fb22"
server_key_iv: "b6398748380f177f91412645f66b2b9e"
server_key_version: "encryptionkey@0"
I20260926 07:44:57.551191 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03"
format_stamp: "Formatted at 2026-09-26 07:44:57 on dist-test-slave-x492"
server_key: "12eebeb98588a22435c3a464e448fb22"
server_key_iv: "b6398748380f177f91412645f66b2b9e"
server_key_version: "encryptionkey@0"
I20260926 07:44:57.556517 1010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260926 07:44:57.559926 1025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:57.561319 1010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:44:57.561491 1010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03"
format_stamp: "Formatted at 2026-09-26 07:44:57 on dist-test-slave-x492"
server_key: "12eebeb98588a22435c3a464e448fb22"
server_key_iv: "b6398748380f177f91412645f66b2b9e"
server_key_version: "encryptionkey@0"
I20260926 07:44:57.561682 1010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:57.577057 1010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:57.581004 1010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:57.581329 1010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:57.599437 1010 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:41945
I20260926 07:44:57.599479 1076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:41945 every 8 connection(s)
I20260926 07:44:57.600817 1010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:57.601075 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1010
I20260926 07:44:57.601289 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:57.601962 11301 external_mini_cluster.cc:1428] Setting key 38c49493afa2880e1fe98e4ece62d108
I20260926 07:44:57.605676 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:57.617013 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:57.603842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59711 (local address 127.11.9.126:41945)
0926 07:44:57.604614 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:57.604629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:57.604677 (+ 48us) server_negotiation.cc:407] Connection header received
0926 07:44:57.606994 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:57.607020 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:57.607492 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:57.607941 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:57.608424 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:57.610465 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:57.611821 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:57.612833 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:57.612997 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:57.614209 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:57.614223 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:57.614240 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:57.614962 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:57.615054 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:57.615091 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:57.615213 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:57.615405 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 07:44:57.615691 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":324,"threads_started":1}
I20260926 07:44:57.619351 1077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03: Bootstrap starting.
I20260926 07:44:57.620667 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:57.622546 1077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:57.623749 1077 log.cc:824] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:57.626950 1077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03: No bootstrap required, opened a new log
I20260926 07:44:57.631736 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:57.625132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34777 (local address 127.11.9.126:41945)
0926 07:44:57.625316 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:57.625322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:57.625812 (+ 490us) server_negotiation.cc:407] Connection header received
0926 07:44:57.625963 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:57.625968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:57.626040 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:57.626161 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:57.626663 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:57.627740 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:57.629040 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:44:57.629784 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:44:57.629928 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:44:57.631010 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:57.631021 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:57.631026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:57.631048 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:57.631097 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:57.631121 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:57.631217 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:57.631341 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 07:44:57.631427 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:44:57.635968 1077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } }
I20260926 07:44:57.636602 1077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:57.636734 1077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbfff3d5c4c4227b1f1d486f5e09f03, State: Initialized, Role: FOLLOWER
I20260926 07:44:57.637461 1077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } }
I20260926 07:44:57.637722 1077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:57.637880 1077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:57.638088 1077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:57.640559 1077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } }
I20260926 07:44:57.641127 1077 leader_election.cc:302] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cbfff3d5c4c4227b1f1d486f5e09f03; no voters:
I20260926 07:44:57.641831 1077 leader_election.cc:288] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:57.641937 1087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:57.642735 1087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbfff3d5c4c4227b1f1d486f5e09f03, State: Running, Role: LEADER
I20260926 07:44:57.643285 1087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } }
I20260926 07:44:57.644097 1077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:57.646407 1089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cbfff3d5c4c4227b1f1d486f5e09f03. Latest consensus state: current_term: 1 leader_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } } }
I20260926 07:44:57.646277 1088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbfff3d5c4c4227b1f1d486f5e09f03" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41945 } } }
I20260926 07:44:57.646979 1089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:57.647050 1088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:57.648778 1096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:57.654112 1096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:57.662591 1096 catalog_manager.cc:1347] Generated new cluster ID: f51ee80c9f234012a17ffdab8716bb67
I20260926 07:44:57.662698 1096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:57.687175 1096 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:57.688460 1096 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:57.696444 1096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cbfff3d5c4c4227b1f1d486f5e09f03: Generated new TSK 0
I20260926 07:44:57.697314 1096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:57.710543 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1010
W20260926 07:44:57.711608 1072 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59711
W20260926 07:44:57.712318 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1010: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:57.712467 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1010
2026-09-26T07:44:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1898 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:57 dist-test-slave-x492 krb5kdc[1108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:57 dist-test-slave-x492 krb5kdc[1108](info): set up 2 sockets
Sep 26 07:44:57 dist-test-slave-x492 krb5kdc[1108](info): commencing operation
krb5kdc: starting...
W20260926 07:44:58.817790 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:44:58 dist-test-slave-x492 krb5kdc[1108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:44:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:44:59.064612 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:41069
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:41069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:44:59.388538 1121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:44:59.388827 1121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:44:59.388917 1121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:44:59.396734 1121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:44:59.396836 1121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:44:59.396881 1121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:44:59.396944 1121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:44:59.396989 1121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:44:59.406847 1121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:41069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:41069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:44:59.408010 1121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:44:59.409268 1121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:44:59.420835 1129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:59.420766 1127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:44:59.420758 1126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:44:59.421648 1121 server_base.cc:1034] running on GCE node
I20260926 07:44:59.422501 1121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:44:59.423817 1121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:44:59.425051 1121 hybrid_clock.cc:648] HybridClock initialized: now 1790408699425015 us; error 47 us; skew 500 ppm
I20260926 07:44:59.425443 1121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:44:59 dist-test-slave-x492 krb5kdc[1108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408699, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:44:59.432241 1121 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:44:59.433785 1121 webserver.cc:460] Webserver started at http://127.11.9.126:32889/ using document root <none> and password file <none>
I20260926 07:44:59.434459 1121 fs_manager.cc:359] Metadata directory not provided
I20260926 07:44:59.434603 1121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:44:59.434911 1121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:44:59.437920 1121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "a288b9bdfb7341939639f0e4a80900b8"
format_stamp: "Formatted at 2026-09-26 07:44:59 on dist-test-slave-x492"
server_key: "b534e181c6a7f4889c9a7d39458d95a6"
server_key_iv: "e38af0038c5dca9506bc24d495aa3c9a"
server_key_version: "encryptionkey@0"
I20260926 07:44:59.438658 1121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "a288b9bdfb7341939639f0e4a80900b8"
format_stamp: "Formatted at 2026-09-26 07:44:59 on dist-test-slave-x492"
server_key: "b534e181c6a7f4889c9a7d39458d95a6"
server_key_iv: "e38af0038c5dca9506bc24d495aa3c9a"
server_key_version: "encryptionkey@0"
I20260926 07:44:59.443573 1121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:44:59.446907 1136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:44:59.448192 1121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:44:59.448381 1121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "a288b9bdfb7341939639f0e4a80900b8"
format_stamp: "Formatted at 2026-09-26 07:44:59 on dist-test-slave-x492"
server_key: "b534e181c6a7f4889c9a7d39458d95a6"
server_key_iv: "e38af0038c5dca9506bc24d495aa3c9a"
server_key_version: "encryptionkey@0"
I20260926 07:44:59.448577 1121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:44:59.454479 1121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:44:59.459189 1121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:44:59.459483 1121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:44:59.477825 1121 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:41069
I20260926 07:44:59.477862 1187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:41069 every 8 connection(s)
I20260926 07:44:59.479168 1121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:44:59.482746 1188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:44:59.487952 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1121
I20260926 07:44:59.488112 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:44:59.488610 11301 external_mini_cluster.cc:1428] Setting key 9f1ecbabec8ddea2b6b057136fa7bf8c
I20260926 07:44:59.493758 1188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8: Bootstrap starting.
I20260926 07:44:59.496191 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:59.490434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40591 (local address 127.11.9.126:41069)
0926 07:44:59.491201 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:59.491215 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:59.491256 (+ 41us) server_negotiation.cc:407] Connection header received
0926 07:44:59.492884 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:59.492906 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:59.493293 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:59.493666 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:59.494735 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:59.494740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:59.494753 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:59.494771 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:59.494819 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:59.494837 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:59.495062 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:59.495219 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 07:44:59.495388 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":343,"threads_started":1}
I20260926 07:44:59.496951 1188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8: Neither blocks nor log segments found. Creating new log.
I20260926 07:44:59.498060 1188 log.cc:824] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:44:59.499789 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:44:59.500829 1188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8: No bootstrap required, opened a new log
I20260926 07:44:59.506457 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:44:59.503876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41997 (local address 127.11.9.126:41069)
0926 07:44:59.504068 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 07:44:59.504077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:44:59.504535 (+ 458us) server_negotiation.cc:407] Connection header received
0926 07:44:59.504689 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:44:59.504692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:44:59.504747 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:44:59.504824 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:44:59.505783 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:44:59.505787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:44:59.505790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 07:44:59.505804 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:44:59.505845 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 07:44:59.505859 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:44:59.505956 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:44:59.506085 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 07:44:59.506131 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 07:44:59.508836 1188 raft_consensus.cc:348] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } }
I20260926 07:44:59.509198 1188 raft_consensus.cc:374] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:44:59.509315 1188 raft_consensus.cc:729] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a288b9bdfb7341939639f0e4a80900b8, State: Initialized, Role: FOLLOWER
I20260926 07:44:59.509896 1188 consensus_queue.cc:260] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } }
I20260926 07:44:59.510130 1188 raft_consensus.cc:388] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 07:44:59.510281 1188 raft_consensus.cc:482] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 07:44:59.510429 1188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:44:59.512584 1188 raft_consensus.cc:504] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } }
I20260926 07:44:59.512989 1188 leader_election.cc:302] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a288b9bdfb7341939639f0e4a80900b8; no voters:
I20260926 07:44:59.513545 1188 leader_election.cc:288] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 07:44:59.513718 1198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:44:59.514984 1198 raft_consensus.cc:686] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [term 1 LEADER]: Becoming Leader. State: Replica: a288b9bdfb7341939639f0e4a80900b8, State: Running, Role: LEADER
I20260926 07:44:59.515579 1188 sys_catalog.cc:564] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:44:59.515507 1198 consensus_queue.cc:237] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } }
I20260926 07:44:59.517916 1199 sys_catalog.cc:455] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a288b9bdfb7341939639f0e4a80900b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } } }
I20260926 07:44:59.518302 1199 sys_catalog.cc:458] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:59.519124 1205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:44:59.519011 1200 sys_catalog.cc:455] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a288b9bdfb7341939639f0e4a80900b8. Latest consensus state: current_term: 1 leader_uuid: "a288b9bdfb7341939639f0e4a80900b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a288b9bdfb7341939639f0e4a80900b8" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 41069 } } }
I20260926 07:44:59.520148 1200 sys_catalog.cc:458] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8 [sys.catalog]: This master's current role is: LEADER
I20260926 07:44:59.524374 1205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:44:59.536132 1205 catalog_manager.cc:1347] Generated new cluster ID: e69f44b1b4fb44f380dbf0c9e5d8fed3
I20260926 07:44:59.536253 1205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:44:59.562625 1205 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:44:59.563050 1205 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:44:59.578552 1205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a288b9bdfb7341939639f0e4a80900b8: Generated new TSK 0
I20260926 07:44:59.579336 1205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:44:59.592626 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1121
W20260926 07:44:59.593981 1183 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40591
W20260926 07:44:59.594666 11301 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1121: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 07:44:59.594810 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1121
2026-09-26T07:44:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1881 ms)
[----------] 18 tests from AuthTokenIssuingTest (34515 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:44:59 dist-test-slave-x492 krb5kdc[1219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:44:59 dist-test-slave-x492 krb5kdc[1219](info): set up 2 sockets
Sep 26 07:44:59 dist-test-slave-x492 krb5kdc[1219](info): commencing operation
krb5kdc: starting...
W20260926 07:45:00.705899 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:45:00 dist-test-slave-x492 krb5kdc[1219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:45:00.954979 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35513
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:45:01.280405 1232 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:01.280776 1232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:01.280912 1232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:01.281044 1232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:45:01.288465 1232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:01.288576 1232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:01.288698 1232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:01.288781 1232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:01.288895 1232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:01.298591 1232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:01.299835 1232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:01.301091 1232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:01.312469 1237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:01.312461 1238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:01.312318 1240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:01.313585 1232 server_base.cc:1034] running on GCE node
I20260926 07:45:01.314320 1232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:01.315590 1232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:01.316831 1232 hybrid_clock.cc:648] HybridClock initialized: now 1790408701316759 us; error 48 us; skew 500 ppm
I20260926 07:45:01.317142 1232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:01 dist-test-slave-x492 krb5kdc[1219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:01.320190 1232 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:45:01.321655 1232 webserver.cc:460] Webserver started at http://127.11.9.126:34771/ using document root <none> and password file <none>
I20260926 07:45:01.322310 1232 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:01.322436 1232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:01.322745 1232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:01.325631 1232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "0e33578637ce453bb685e3210facabe6"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "d4a29e4620b58b34182457705ecd38bf"
server_key_iv: "154a248b2f2739bc4b384fc7bc42aee8"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.326403 1232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "0e33578637ce453bb685e3210facabe6"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "d4a29e4620b58b34182457705ecd38bf"
server_key_iv: "154a248b2f2739bc4b384fc7bc42aee8"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.331403 1232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 07:45:01.334769 1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:01.336019 1232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 07:45:01.336199 1232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "0e33578637ce453bb685e3210facabe6"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "d4a29e4620b58b34182457705ecd38bf"
server_key_iv: "154a248b2f2739bc4b384fc7bc42aee8"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.336409 1232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:01.355216 1232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:01.359035 1232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:01.359293 1232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:01.377298 1232 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:35513
I20260926 07:45:01.377339 1298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:35513 every 8 connection(s)
I20260926 07:45:01.378583 1232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:45:01.382131 1299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:01.388566 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1232
I20260926 07:45:01.388855 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:45:01.388422 1299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:01.389654 11301 external_mini_cluster.cc:1428] Setting key fe88b46c0a9fa11e320e7d5a74e71295
Sep 26 07:45:01 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:45:01.410135 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.391505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.11.9.126:35513)
0926 07:45:01.393102 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:01.393110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:01.393157 (+ 47us) server_negotiation.cc:407] Connection header received
0926 07:45:01.394661 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:01.394666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:01.394888 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:01.395091 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:01.396339 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.397569 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.398913 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.399447 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.399652 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.402933 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:01.402941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:01.402952 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:01.402982 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:01.405770 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.406449 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.406463 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.406466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.406603 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.406993 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.407001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.407004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.407475 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:01.407678 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:01.408204 (+ 526us) server_negotiation.cc:299] Negotiation successful
0926 07:45:01.408588 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":1383,"thread_start_us":1193,"threads_started":1}
I20260926 07:45:01.410115 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.391505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35513 (local address 127.0.0.1:49586)
0926 07:45:01.392330 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.392373 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:01.394311 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:01.395102 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:01.395108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:01.395821 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:01.396130 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.396146 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.397718 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.397723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.398777 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.398788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.399669 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.399676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.399855 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.400505 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:01.400526 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:01.402703 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:01.405910 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.405922 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.405933 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.406314 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.406720 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.406724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.406728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.406888 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.407616 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:01.407623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:01.407843 (+ 220us) client_negotiation.cc:769] Sending connection context
0926 07:45:01.408090 (+ 247us) client_negotiation.cc:240] Negotiation successful
0926 07:45:01.408697 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":142,"thread_start_us":367,"threads_started":1}
WARNING: no policy specified for kudu/127.11.9.125@KRBTEST.COM; defaulting to no policy
I20260926 07:45:01.413439 1299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:01.414429 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.414048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33798)
0926 07:45:01.414222 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.414323 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.125@KRBTEST.COM" created.
W20260926 07:45:01.414706 1251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
I20260926 07:45:01.416491 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.416034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33810)
0926 07:45:01.416217 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.416308 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 07:45:01.416769 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:44967: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
I20260926 07:45:01.445402 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:44967
--webserver_interface=127.11.9.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:45:01.466128 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } attempt: 1
I20260926 07:45:01.467422 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.466924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33824)
0926 07:45:01.467163 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.467290 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 07:45:01.468681 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.468267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33838)
0926 07:45:01.468461 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.468530 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260926 07:45:01.468971 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:44967: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
I20260926 07:45:01.537295 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } attempt: 2
I20260926 07:45:01.538460 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.538017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33852)
0926 07:45:01.538205 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.538308 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 07:45:01.539459 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.539123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33860)
0926 07:45:01.539270 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.539345 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 07:45:01.539705 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:44967: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
I20260926 07:45:01.630970 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } attempt: 3
I20260926 07:45:01.632213 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.631773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33866)
0926 07:45:01.631987 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.632077 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 07:45:01.633285 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.632855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33870)
0926 07:45:01.633001 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.633077 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 07:45:01.633472 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:44967: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
W20260926 07:45:01.771318 1306 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:01.771801 1306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:01.771974 1306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:01.772122 1306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 07:45:01.776736 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } attempt: 4
I20260926 07:45:01.778019 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.777448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33886)
0926 07:45:01.777801 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.777887 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260926 07:45:01.779023 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.778667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33902)
0926 07:45:01.778814 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.778904 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 07:45:01.779255 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:44967: Network error: Client connection negotiation failed: client connection to 127.11.9.125:44967: connect: Connection refused (error 111)
W20260926 07:45:01.779736 1306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:01.779811 1306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:01.779903 1306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:01.779969 1306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:01.780062 1306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:01.790016 1306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:44967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.9.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:01.791265 1306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:01.792569 1306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:01.804119 1311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:01.804147 1314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:01.804490 1312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:01.805421 1306 server_base.cc:1034] running on GCE node
I20260926 07:45:01.806193 1306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:01.807575 1306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:01.808835 1306 hybrid_clock.cc:648] HybridClock initialized: now 1790408701808773 us; error 35 us; skew 500 ppm
I20260926 07:45:01.809237 1306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:01 dist-test-slave-x492 krb5kdc[1219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:01.812572 1306 init.cc:377] Logged in from keytab as kudu/127.11.9.125@KRBTEST.COM (short username kudu)
I20260926 07:45:01.814041 1306 webserver.cc:460] Webserver started at http://127.11.9.125:36745/ using document root <none> and password file <none>
I20260926 07:45:01.814688 1306 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:01.814776 1306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:01.815037 1306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:01.818192 1306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data/instance:
uuid: "043880fa3f634bb5bfe175040eee20e8"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "2662ba0bc88f4a561e06885333dce472"
server_key_iv: "e3f60259a429ec42fdc508828e78f751"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.818955 1306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal/instance:
uuid: "043880fa3f634bb5bfe175040eee20e8"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "2662ba0bc88f4a561e06885333dce472"
server_key_iv: "e3f60259a429ec42fdc508828e78f751"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.824110 1306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 07:45:01.827783 1321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:01.829387 1306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:45:01.829560 1306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "043880fa3f634bb5bfe175040eee20e8"
format_stamp: "Formatted at 2026-09-26 07:45:01 on dist-test-slave-x492"
server_key: "2662ba0bc88f4a561e06885333dce472"
server_key_iv: "e3f60259a429ec42fdc508828e78f751"
server_key_version: "encryptionkey@0"
I20260926 07:45:01.829772 1306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:01.855285 1306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:01.859845 1306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:01.860123 1306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:01.878784 1306 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.125:44967
I20260926 07:45:01.878885 1372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.125:44967 every 8 connection(s)
I20260926 07:45:01.880125 1306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
I20260926 07:45:01.883908 1373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:01.889503 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1306
I20260926 07:45:01.889698 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-1/wal/instance
I20260926 07:45:01.890115 11301 external_mini_cluster.cc:1428] Setting key 0c489021e2a5607c342ca27919f6ce58
I20260926 07:45:01.890977 1373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:01 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:45:01.913617 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.893717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.11.9.126:35513)
0926 07:45:01.893949 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:01.893956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:01.895148 (+ 1192us) server_negotiation.cc:407] Connection header received
0926 07:45:01.897145 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:01.897152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:01.897279 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:01.897373 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:01.899571 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.900246 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.902865 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.903328 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.903427 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.907876 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:01.907881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:01.907885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:01.907915 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:01.910623 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.911342 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.911348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.911356 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.911473 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.911902 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.911908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.911912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.912200 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:01.912354 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:01.913028 (+ 674us) server_negotiation.cc:299] Negotiation successful
0926 07:45:01.913289 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
WARNING: no policy specified for kudu/127.11.9.124@KRBTEST.COM; defaulting to no policy
I20260926 07:45:01.914565 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.894162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35513 (local address 127.0.0.1:49598)
0926 07:45:01.894963 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.895010 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:01.896863 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:01.897690 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:01.897704 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:01.898552 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:01.899372 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.899400 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.900409 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.900419 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.902700 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.902718 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.903571 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.903582 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.903911 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.904651 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:01.904686 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:01.907642 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:01.910768 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.910776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.910792 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.911212 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.911583 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.911588 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.911592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.911787 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.912327 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:01.912335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:01.912656 (+ 321us) client_negotiation.cc:769] Sending connection context
0926 07:45:01.912932 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 07:45:01.913458 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":339,"threads_started":1}
I20260926 07:45:01.916374 1373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.124@KRBTEST.COM" created.
Sep 26 07:45:01 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:01.932405 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.917309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33914)
0926 07:45:01.917605 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.917635 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:01.917807 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:01.918495 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:01.918503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:01.918998 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:01.919205 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.919220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.920546 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.920554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.921732 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:01.921743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.922525 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.922530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:01.922704 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.923269 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:01.923287 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:01.925568 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:01.929880 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.929884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.929887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.930338 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.930794 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:01.930798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:01.930800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:01.930938 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:01.931691 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:01.931696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:01.931809 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 07:45:01.931925 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 07:45:01.932132 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260926 07:45:01.932910 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.917453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.11.9.125:44967)
0926 07:45:01.918080 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:01.918086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:01.918110 (+ 24us) server_negotiation.cc:407] Connection header received
0926 07:45:01.918151 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:01.918157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:01.918326 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:01.918435 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:01.919384 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.920404 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.921869 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:01.922385 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:01.922519 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:01.925784 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:01.925797 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:01.925816 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:01.925866 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:01.929728 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.930490 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.930501 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.930507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.930659 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:01.931075 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:01.931082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:01.931088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:01.931555 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:01.931767 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:01.932158 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 07:45:01.932445 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
I20260926 07:45:01.935442 1373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:01.936558 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.936102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50834)
0926 07:45:01.936301 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.936412 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 07:45:01.936846 1325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:01.938452 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.938127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50846)
0926 07:45:01.938276 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.938355 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 07:45:01.938683 1373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
I20260926 07:45:01.951988 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:39589
--webserver_interface=127.11.9.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:45:01.987968 1373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 1
I20260926 07:45:01.989246 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.988696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50858)
0926 07:45:01.988919 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.989041 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 07:45:01.990352 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:01.989984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50872)
0926 07:45:01.990142 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:01.990227 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 07:45:01.990566 1373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.058799 1373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 2
I20260926 07:45:02.060230 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.059666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50880)
0926 07:45:02.059927 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.060039 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 07:45:02.061331 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.060916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50886)
0926 07:45:02.061065 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.061149 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 07:45:02.061535 1373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.078531 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } attempt: 5
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:02.094099 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.079238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33916)
0926 07:45:02.079452 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.079491 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.079633 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.080047 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.080052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.080553 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.080833 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.080849 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.081997 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.082002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.083371 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.083387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.084108 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.084114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.084349 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.084989 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.085018 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.087811 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.091884 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.091888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.091890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.092245 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.092623 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.092626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.092628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.092766 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.093385 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.093388 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.093515 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.093633 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.093854 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 07:45:02.094375 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.079325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.11.9.125:44967)
0926 07:45:02.079501 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.079505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.079572 (+ 67us) server_negotiation.cc:407] Connection header received
0926 07:45:02.079794 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.079817 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.079890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.079967 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.081005 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.081859 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.083521 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.083939 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.084031 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.088066 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.088074 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.088081 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.088129 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.091744 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.092368 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.092373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.092377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.092517 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.092881 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.092885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.092888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.093261 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.093409 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.093790 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.094058 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:45:02.095377 1299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:02.096396 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.096020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50896)
0926 07:45:02.096172 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.096255 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 07:45:02.097443 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.097120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50912)
0926 07:45:02.097282 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.097354 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260926 07:45:02.097707 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.148954 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 1
I20260926 07:45:02.150163 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.149709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50924)
0926 07:45:02.149904 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.149994 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 07:45:02.151285 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.150948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50936)
0926 07:45:02.151095 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.151158 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 07:45:02.151499 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.152755 1373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 3
I20260926 07:45:02.153808 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.153327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50952)
0926 07:45:02.153525 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.153637 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 07:45:02.154841 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.154406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50960)
0926 07:45:02.154624 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.154712 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260926 07:45:02.155046 1373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.219763 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 2
I20260926 07:45:02.220942 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.220458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50966)
0926 07:45:02.220701 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.220791 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 07:45:02.222113 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.221726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50968)
0926 07:45:02.221930 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.222003 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260926 07:45:02.222370 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
W20260926 07:45:02.281598 1380 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:02.281947 1380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:02.282066 1380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:02.282197 1380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:45:02.289244 1380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:02.289320 1380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:02.289446 1380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:02.289512 1380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:02.289666 1380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:02.298270 1373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 4
I20260926 07:45:02.299477 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.298929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50970)
0926 07:45:02.299174 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.299286 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 07:45:02.299463 1380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:35513,127.11.9.125:44967,127.11.9.124:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:39589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.9.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:02.300539 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.300171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50980)
0926 07:45:02.300327 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.300421 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 07:45:02.300732 1373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.301149 1380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:02.302534 1380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:02.314072 1385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:02.314101 1386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:02.314079 1388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:02.315133 1380 server_base.cc:1034] running on GCE node
I20260926 07:45:02.315794 1380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:02.316975 1380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:02.318181 1380 hybrid_clock.cc:648] HybridClock initialized: now 1790408702318118 us; error 35 us; skew 500 ppm
I20260926 07:45:02.318447 1380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408702, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:02.321457 1380 init.cc:377] Logged in from keytab as kudu/127.11.9.124@KRBTEST.COM (short username kudu)
I20260926 07:45:02.322773 1380 webserver.cc:460] Webserver started at http://127.11.9.124:41987/ using document root <none> and password file <none>
I20260926 07:45:02.323438 1380 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:02.323546 1380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:02.323853 1380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:02.326774 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data/instance:
uuid: "b2225d315cde46fdbd3a72f272956e55"
format_stamp: "Formatted at 2026-09-26 07:45:02 on dist-test-slave-x492"
server_key: "d989ab0073dcbe58f368de6868d8d54e"
server_key_iv: "5a94c82df47a1bdc99f39beee666c853"
server_key_version: "encryptionkey@0"
I20260926 07:45:02.327539 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal/instance:
uuid: "b2225d315cde46fdbd3a72f272956e55"
format_stamp: "Formatted at 2026-09-26 07:45:02 on dist-test-slave-x492"
server_key: "d989ab0073dcbe58f368de6868d8d54e"
server_key_iv: "5a94c82df47a1bdc99f39beee666c853"
server_key_version: "encryptionkey@0"
I20260926 07:45:02.328629 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 3
I20260926 07:45:02.329741 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.329296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:50994)
0926 07:45:02.329498 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.329578 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 07:45:02.330791 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.330431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:51002)
0926 07:45:02.330575 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.330679 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 07:45:02.330998 1299 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:39589: Network error: Client connection negotiation failed: client connection to 127.11.9.124:39589: connect: Connection refused (error 111)
I20260926 07:45:02.333103 1380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 07:45:02.336601 1395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:02.337903 1380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:45:02.338074 1380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "b2225d315cde46fdbd3a72f272956e55"
format_stamp: "Formatted at 2026-09-26 07:45:02 on dist-test-slave-x492"
server_key: "d989ab0073dcbe58f368de6868d8d54e"
server_key_iv: "5a94c82df47a1bdc99f39beee666c853"
server_key_version: "encryptionkey@0"
I20260926 07:45:02.338297 1380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:02.358711 1380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:02.362804 1380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:02.363018 1380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:02.381425 1380 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.124:39589
I20260926 07:45:02.381476 1446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.124:39589 every 8 connection(s)
I20260926 07:45:02.382692 1380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
I20260926 07:45:02.385097 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1380
I20260926 07:45:02.385334 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790408351541924-11301-0/minicluster-data/master-2/wal/instance
I20260926 07:45:02.385778 11301 external_mini_cluster.cc:1428] Setting key f3a3812a59f69472d942f44242f2ff64
I20260926 07:45:02.387039 1447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:02.395862 1447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408702, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:45:02.408888 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.387513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.11.9.124:39589)
0926 07:45:02.388263 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.388273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.388311 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:45:02.389923 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.389943 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.390286 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.390681 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.392358 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.393742 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.395439 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.396086 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.396215 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.400745 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.400757 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.400771 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.400822 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.404474 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.405254 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.405263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.405271 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.405472 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.405833 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.405837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.405840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.406459 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.406668 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.407191 (+ 523us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.407561 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":347,"threads_started":1}
I20260926 07:45:02.413954 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.398505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.11.9.126:35513)
0926 07:45:02.398719 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.398725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.399329 (+ 604us) server_negotiation.cc:407] Connection header received
0926 07:45:02.399504 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.399511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.399591 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.399697 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.400850 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.401603 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.403708 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.404108 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.404200 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.407723 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.407729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.407735 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.407774 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.410854 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.411800 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.411805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.411808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.412043 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.412500 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.412510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.412515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.412883 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.413029 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.413542 (+ 513us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.413701 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:45:02.414247 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.398438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:35513 (local address 127.0.0.1:49610)
0926 07:45:02.399195 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.399233 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.399375 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.399797 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.399805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.400390 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.400707 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.400718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.401752 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.401760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.403554 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.403569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.404294 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.404301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.404483 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.405069 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.405089 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.407541 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.411014 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.411022 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.411149 (+ 127us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.411655 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.412170 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.412175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.412178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.412358 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.413044 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.413051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.413254 (+ 203us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.413461 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.413808 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":399,"threads_started":1}
I20260926 07:45:02.415748 1447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408702, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:02.427961 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.412090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.11.9.126:35513)
0926 07:45:02.412763 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.412768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.412790 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:45:02.412834 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.412838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.412904 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.412999 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.414572 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.415781 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.417334 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.417767 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.417890 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.422374 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.422379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.422383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.422410 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.425363 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.426009 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.426014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.426019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.426131 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.426504 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.426509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.426514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.426775 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.427066 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.427359 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.427569 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":293,"threads_started":1}
I20260926 07:45:02.428551 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.416459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:44967 (local address 127.0.0.1:33922)
0926 07:45:02.416627 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.416644 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.416732 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.416996 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.417000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.417443 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.417565 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.417573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.418626 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.418635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.420128 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.420147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.420803 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.420811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.420988 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.421480 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.421496 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.423405 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.426317 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.426323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.426327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.426660 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.427041 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.427046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.427049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.427189 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.427690 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.427694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.427832 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.427965 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.428223 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 07:45:02.428606 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.416587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.11.9.125:44967)
0926 07:45:02.416743 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.416747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.416763 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:45:02.416818 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.416822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.416875 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.416952 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.417709 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.418490 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.420277 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.420652 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.420739 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.423587 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.423599 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.423604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.423636 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.426187 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.426787 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.426793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.426796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.426930 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.427301 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.427307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.427311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.427557 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.427710 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.428051 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.428248 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:45:02.429630 1447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408702, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:02.442792 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.430225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:51024)
0926 07:45:02.430381 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.430397 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.430479 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.430870 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.430874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.431202 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.431336 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.431343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.432277 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.432281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.433374 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.433387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.434269 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.434274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.434489 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.434930 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.434956 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.437285 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.440377 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.440384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.440388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.440735 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.441081 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.441087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.441093 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.441278 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.441767 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.441771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.441914 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.442057 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.442354 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":4}
I20260926 07:45:02.442878 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.430398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.11.9.124:39589)
0926 07:45:02.430553 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.430559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.430579 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:45:02.430625 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.430631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.430702 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.430807 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.431492 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.432144 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.433523 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.433985 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.434102 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.437434 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.437438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.437441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.437466 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.440239 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.440844 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.440851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.440856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.440979 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.441386 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.441391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.441394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.441651 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.441798 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.442151 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.442543 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":39}
I20260926 07:45:02.444094 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.429783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.11.9.125:44967)
0926 07:45:02.429945 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.429948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.429962 (+ 14us) server_negotiation.cc:407] Connection header received
0926 07:45:02.429997 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.430000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.430046 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.430155 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.431456 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.432140 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.433326 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.433910 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.434000 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.438690 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.438695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.438698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.438727 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.441172 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.441970 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.441977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.441980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.442115 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.442557 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.442562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.442565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.442970 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.443236 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.443452 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.443605 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 07:45:02.448792 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:45:02.449111 1447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: Bootstrap starting.
I20260926 07:45:02.452817 1447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:02.453924 1447 log.cc:824] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:02.452077 1274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0e33578637ce453bb685e3210facabe6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49614
I20260926 07:45:02.456573 1447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: No bootstrap required, opened a new log
I20260926 07:45:02.463378 1447 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.463711 1447 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2225d315cde46fdbd3a72f272956e55, State: Initialized, Role: FOLLOWER
I20260926 07:45:02.464115 1447 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.465116 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.465498 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.466034 1447 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:45:02.474885 1465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:45:02.475034 1465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:45:02.473143 1380 master_runner.cc:186] Error getting master registration for 127.11.9.126:35513: OK, instance_id { permanent_uuid: "0e33578637ce453bb685e3210facabe6" instance_seqno: 1790408701372056 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:45:02.484302 1380 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:45:02.508219 1299 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 4
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
I20260926 07:45:02.524183 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.509072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:51030)
0926 07:45:02.509299 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.509320 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.509462 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.509827 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.509831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.510258 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.510412 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.510421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.511887 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.511892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.512794 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.512806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.513830 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.513835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.514026 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.514730 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.514762 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.517610 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.521473 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.521479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.521483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.522002 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.522404 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.522410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.522414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.522578 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.523178 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.523183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.523376 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.523527 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.523807 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 07:45:02.524219 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.509111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.11.9.124:39589)
0926 07:45:02.509320 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.509326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.509380 (+ 54us) server_negotiation.cc:407] Connection header received
0926 07:45:02.509596 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.509601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.509694 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.509793 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.510563 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.511753 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.512937 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.513680 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.513813 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.517789 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.517802 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.517807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.517837 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.521337 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.522133 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.522138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.522141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.522289 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.522697 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.522702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.522705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.523054 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.523231 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.523639 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.523859 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 07:45:02.528983 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: Bootstrap starting.
I20260926 07:45:02.532246 1299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:02.533440 1299 log.cc:824] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:02.536234 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: No bootstrap required, opened a new log
I20260926 07:45:02.541596 1299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.541970 1299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e33578637ce453bb685e3210facabe6, State: Initialized, Role: FOLLOWER
I20260926 07:45:02.542394 1299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.543388 1468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.543749 1468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.544476 1299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:45:02.544641 1274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 07:45:02.544867 1274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:02.548445 1274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.551011 1274 leader_election.cc:288] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [CANDIDATE]: Term 1 election: Requested vote from peers 043880fa3f634bb5bfe175040eee20e8 (127.11.9.125:44967), b2225d315cde46fdbd3a72f272956e55 (127.11.9.124:39589)
I20260926 07:45:02.550987 1422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e33578637ce453bb685e3210facabe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b2225d315cde46fdbd3a72f272956e55"
I20260926 07:45:02.551450 1422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:02.554936 1422 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e33578637ce453bb685e3210facabe6 in term 1.
I20260926 07:45:02.555511 1251 leader_election.cc:302] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e33578637ce453bb685e3210facabe6, b2225d315cde46fdbd3a72f272956e55; no voters:
I20260926 07:45:02.555933 1468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:45:02.567181 1468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [term 1 LEADER]: Becoming Leader. State: Replica: 0e33578637ce453bb685e3210facabe6, State: Running, Role: LEADER
I20260926 07:45:02.567807 1468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.575456 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e33578637ce453bb685e3210facabe6. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.575879 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:02.578442 1483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:45:02.588727 1483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:45:02.599937 1373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } attempt: 5
Sep 26 07:45:02 dist-test-slave-x492 krb5kdc[1219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408701, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
W20260926 07:45:02.611937 1253 tablet.cc:2369] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:45:02.614989 1401 tablet.cc:2369] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:45:02.627759 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.600545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:39589 (local address 127.0.0.1:51036)
0926 07:45:02.600925 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:02.600945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:02.601057 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:02.601993 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:02.602000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:02.602515 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:02.602716 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.602728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.604218 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.604223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.605674 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:02.605688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.606858 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.606865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:02.607097 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.607800 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:02.607823 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:02.611086 (+ 3263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:02.620436 (+ 9350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.620468 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.620475 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.621608 (+ 1133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.622555 (+ 947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:02.622563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:02.622597 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:02.622960 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:02.625552 (+ 2592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:02.625585 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:02.625884 (+ 299us) client_negotiation.cc:769] Sending connection context
0926 07:45:02.626197 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 07:45:02.626802 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260926 07:45:02.628175 1422 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [term 1 FOLLOWER]: Refusing update from remote peer 0e33578637ce453bb685e3210facabe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:45:02.631616 1468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:45:02.632061 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.601325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.11.9.124:39589)
0926 07:45:02.601509 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.601514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.601533 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:45:02.601593 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.601598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.601674 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.601808 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.602925 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.604062 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.605837 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.606553 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.606668 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.611459 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.611468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.611476 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.611527 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.620176 (+ 8649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.621964 (+ 1788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.621974 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.621980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.622323 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.623430 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.623438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.623444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.625108 (+ 1664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.628716 (+ 3608us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.629359 (+ 643us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.630278 (+ 919us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 07:45:02.640020 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.611536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.11.9.126:35513)
0926 07:45:02.611724 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.611737 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.615823 (+ 4086us) server_negotiation.cc:407] Connection header received
0926 07:45:02.616725 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.616733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.616837 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.616915 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.625614 (+ 8699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.626588 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.628860 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.629574 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.629680 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.632795 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.632803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.632807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.632847 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.636720 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.637660 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.637665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.637669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.637829 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.638286 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.638291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.638295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.638659 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.638934 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.639469 (+ 535us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.639690 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 07:45:02.644748 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: Bootstrap starting.
I20260926 07:45:02.656885 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.629723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.11.9.124:39589)
0926 07:45:02.633086 (+ 3363us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.633095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.640722 (+ 7627us) server_negotiation.cc:407] Connection header received
0926 07:45:02.640951 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.640957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.641028 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.641093 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.642500 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.643509 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.645269 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.645949 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.646043 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.649568 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.649575 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.649581 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.649624 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.653525 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.654404 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.654409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.654413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.654560 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.655020 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.655029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.655035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.655411 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.655657 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.656236 (+ 579us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.656465 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2961,"threads_started":1}
I20260926 07:45:02.635212 1232 master_runner.cc:186] Error getting master registration for 127.11.9.125:44967: OK, instance_id { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" instance_seqno: 1790408701873504 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:45:02.666347 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.616167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.11.9.125:44967)
0926 07:45:02.616642 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.616650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.617102 (+ 452us) server_negotiation.cc:407] Connection header received
0926 07:45:02.617365 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.617383 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.617457 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.617525 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.619717 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.620790 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.622647 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.623286 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.623388 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.658860 (+ 35472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:02.658868 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:02.658873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:02.658928 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:02.662503 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.663315 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.663320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.663326 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.663473 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:02.663887 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:02.663892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:02.663895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:02.664250 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:02.664508 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.665545 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.665831 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 07:45:02.668673 1373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:02.670078 1373 log.cc:824] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:02.665556 1232 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:45:02.672220 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e33578637ce453bb685e3210facabe6. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.672520 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.675699 1488 mvcc.cc:205] Tried to move back new op lower bound from 7333514045847142400 to 7333514045724295168. Current Snapshot: MvccSnapshot[applied={T|T < 7333514045847142400}]
I20260926 07:45:02.676317 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: No bootstrap required, opened a new log
I20260926 07:45:02.689714 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.689967 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.692742 1483 catalog_manager.cc:1347] Generated new cluster ID: eed088a3af6b4c3f94212c9dd0456ea5
I20260926 07:45:02.692847 1483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:45:02.694521 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.694796 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:02.705994 1373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.706453 1373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043880fa3f634bb5bfe175040eee20e8, State: Initialized, Role: FOLLOWER
I20260926 07:45:02.707029 1373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } }
I20260926 07:45:02.717751 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.719259 1373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:45:02.719430 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.719676 1347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:02.720034 1348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e33578637ce453bb685e3210facabe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "043880fa3f634bb5bfe175040eee20e8"
I20260926 07:45:02.723870 1347 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [term 1 FOLLOWER]: Refusing update from remote peer 0e33578637ce453bb685e3210facabe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:45:02.727090 1348 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e33578637ce453bb685e3210facabe6 in term 1.
I20260926 07:45:02.729394 1479 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:45:02.730468 1483 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:45:02.732331 1483 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:45:02.748342 1483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6: Generated new TSK 0
I20260926 07:45:02.749377 1483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:45:02.795413 1498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e33578637ce453bb685e3210facabe6. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.795774 1498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.796120 1507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.796466 1507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:02.797878 1468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e33578637ce453bb685e3210facabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e33578637ce453bb685e3210facabe6" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 35513 } } peers { permanent_uuid: "043880fa3f634bb5bfe175040eee20e8" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 44967 } } peers { permanent_uuid: "b2225d315cde46fdbd3a72f272956e55" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 39589 } } }
I20260926 07:45:02.798183 1468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e33578637ce453bb685e3210facabe6 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:02.884155 1518 catalog_manager.cc:1259] Loaded cluster ID: eed088a3af6b4c3f94212c9dd0456ea5
I20260926 07:45:02.884279 1518 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: loading cluster ID for follower catalog manager: success
I20260926 07:45:02.888093 1518 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: acquiring CA information for follower catalog manager: success
I20260926 07:45:02.890406 1518 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 043880fa3f634bb5bfe175040eee20e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 07:45:02.932114 1452 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:02.906886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.11.9.126:35513)
0926 07:45:02.907084 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.907096 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.921382 (+ 14286us) server_negotiation.cc:407] Connection header received
0926 07:45:02.929365 (+ 7983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.929371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.929461 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.929547 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.931839 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49640: BlockingRecv error: recv got EOF from 127.0.0.1:49640 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260926 07:45:02.934448 1377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:02.907386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33944 (local address 127.11.9.125:44967)
0926 07:45:02.907547 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.907554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.931030 (+ 23476us) server_negotiation.cc:407] Connection header received
0926 07:45:02.932564 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.932571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.932665 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.932754 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.934216 (+ 1462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33944: BlockingRecv error: recv got EOF from 127.0.0.1:33944 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 07:45:02.936860 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:02.921678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.11.9.124:39589)
0926 07:45:02.921919 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.921925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.934909 (+ 12984us) server_negotiation.cc:407] Connection header received
0926 07:45:02.935152 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.935158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.935242 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.935343 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:02.936465 (+ 1122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51056: BlockingRecv error: recv got EOF from 127.0.0.1:51056 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:45:02.990091 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:02.974773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.11.9.125:44967)
0926 07:45:02.974978 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:02.974985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:02.982036 (+ 7051us) server_negotiation.cc:407] Connection header received
0926 07:45:02.982420 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:02.982428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:02.982664 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:02.982864 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:02.983609 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.984784 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.986887 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:02.987547 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:02.987650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:02.988420 (+ 770us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:02.989194 (+ 774us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:02.989339 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:02.989602 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 07:45:02.989721 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 07:45:03.035553 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.026004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.11.9.124:39589)
0926 07:45:03.026213 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.026220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.033398 (+ 7178us) server_negotiation.cc:407] Connection header received
0926 07:45:03.033976 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.033983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.034120 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.034200 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.035321 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51060: BlockingRecv error: recv got EOF from 127.0.0.1:51060 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:45:03.088487 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.068963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.11.9.125:44967)
0926 07:45:03.069207 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.069218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.081745 (+ 12527us) server_negotiation.cc:407] Connection header received
0926 07:45:03.082087 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.082094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.082225 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.082305 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.083069 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.084215 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.086272 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.086906 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.087001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.087597 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.087765 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.087843 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.088075 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.088166 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 07:45:03.133253 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.123966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.11.9.124:39589)
0926 07:45:03.124146 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.124152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.131338 (+ 7186us) server_negotiation.cc:407] Connection header received
0926 07:45:03.131800 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.131806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.131895 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.131968 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.132969 (+ 1001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51068: BlockingRecv error: recv got EOF from 127.0.0.1:51068 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 07:45:03.159629 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.153837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.11.9.125:44967)
0926 07:45:03.154076 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.154085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.154651 (+ 566us) server_negotiation.cc:407] Connection header received
0926 07:45:03.154895 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.154900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.154970 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.155071 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.155503 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.156276 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.157818 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.158253 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.158388 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.158805 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.158914 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.159014 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.159157 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.159277 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 07:45:03.179136 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.170495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.11.9.124:39589)
0926 07:45:03.170674 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.170679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.177764 (+ 7085us) server_negotiation.cc:407] Connection header received
0926 07:45:03.177983 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.177988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.178074 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.178187 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.178946 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51084: BlockingRecv error: recv got EOF from 127.0.0.1:51084 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 07:45:03.193099 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.186838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.11.9.125:44967)
0926 07:45:03.187122 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.187131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.187649 (+ 518us) server_negotiation.cc:407] Connection header received
0926 07:45:03.187915 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.187919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.187986 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.188088 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.188500 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.189503 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.191183 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.191809 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.191958 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.192348 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.192449 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.192546 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.192699 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.192787 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260926 07:45:03.201056 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.198909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.11.9.124:39589)
0926 07:45:03.199193 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.199218 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.199731 (+ 513us) server_negotiation.cc:407] Connection header received
0926 07:45:03.199905 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.199911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.200002 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.200109 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.200827 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51088: BlockingRecv error: recv got EOF from 127.0.0.1:51088 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
I20260926 07:45:03.218508 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.212772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.11.9.125:44967)
0926 07:45:03.213028 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.213036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.213615 (+ 579us) server_negotiation.cc:407] Connection header received
0926 07:45:03.213833 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.213839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.213918 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.214020 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.214442 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.215182 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.216608 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.217110 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.217290 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.217700 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.217816 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.217919 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.218061 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.218146 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260926 07:45:03.231226 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.228849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.11.9.124:39589)
0926 07:45:03.229091 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.229101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.229736 (+ 635us) server_negotiation.cc:407] Connection header received
0926 07:45:03.230006 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.230012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.230100 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.230217 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.230984 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51102: BlockingRecv error: recv got EOF from 127.0.0.1:51102 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 07:45:03.260617 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.255011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.11.9.125:44967)
0926 07:45:03.255232 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.255241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.255809 (+ 568us) server_negotiation.cc:407] Connection header received
0926 07:45:03.256043 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.256048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.256115 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.256223 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.256643 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.257536 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.258835 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.259271 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.259430 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.259868 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.259980 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.260081 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.260237 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.260325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260926 07:45:03.268507 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.266415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.11.9.124:39589)
0926 07:45:03.266632 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.266641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.267125 (+ 484us) server_negotiation.cc:407] Connection header received
0926 07:45:03.267321 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.267344 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.267440 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.267550 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.268266 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51108: BlockingRecv error: recv got EOF from 127.0.0.1:51108 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:45:03.309300 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.303613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.11.9.125:44967)
0926 07:45:03.303838 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.303849 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.304423 (+ 574us) server_negotiation.cc:407] Connection header received
0926 07:45:03.304741 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.304746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.304817 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.304920 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.305421 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.306165 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.307474 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.307914 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.308055 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.308465 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.308600 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.308709 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.308853 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.308936 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 07:45:03.317356 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.314645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.11.9.124:39589)
0926 07:45:03.315319 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.315325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.316049 (+ 724us) server_negotiation.cc:407] Connection header received
0926 07:45:03.316254 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.316259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.316340 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.316443 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.317137 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51120: BlockingRecv error: recv got EOF from 127.0.0.1:51120 (error 108)
Metrics: {"server-negotiator.queue_time_us":510}
I20260926 07:45:03.417943 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.411756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.11.9.125:44967)
0926 07:45:03.411972 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.411979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.412276 (+ 297us) server_negotiation.cc:407] Connection header received
0926 07:45:03.412437 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.412450 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.412552 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.412668 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.413086 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.414380 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.415682 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.416346 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.416480 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.416969 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.417129 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.417264 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.417522 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.417620 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 07:45:03.426362 1492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:03.424187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.11.9.124:39589)
0926 07:45:03.424419 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.424425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.424814 (+ 389us) server_negotiation.cc:407] Connection header received
0926 07:45:03.424981 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.424985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.425068 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.425218 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:03.426101 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51136: BlockingRecv error: recv got EOF from 127.0.0.1:51136 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 07:45:03.476991 1465 catalog_manager.cc:1259] Loaded cluster ID: eed088a3af6b4c3f94212c9dd0456ea5
I20260926 07:45:03.477089 1465 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: loading cluster ID for follower catalog manager: success
I20260926 07:45:03.479548 1465 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: acquiring CA information for follower catalog manager: success
I20260926 07:45:03.481321 1465 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2225d315cde46fdbd3a72f272956e55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:45:03.562961 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.558140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.11.9.125:44967)
0926 07:45:03.558327 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.558332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.558724 (+ 392us) server_negotiation.cc:407] Connection header received
0926 07:45:03.558894 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.558898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.558970 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.559061 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.559466 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.560152 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.561546 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.561946 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.562036 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.562367 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.562479 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.562544 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.562692 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.562750 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:45:03.575312 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.569041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.11.9.124:39589)
0926 07:45:03.569321 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.569329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.569677 (+ 348us) server_negotiation.cc:407] Connection header received
0926 07:45:03.569924 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.569929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.570111 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.570249 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.570701 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.571480 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.572910 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.573433 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.573595 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.574141 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.574691 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.574862 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.574927 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.575011 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:45:03.588203 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.581193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.11.9.126:35513)
0926 07:45:03.582647 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.582651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.582666 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:45:03.582753 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.582756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.582904 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.583075 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.583774 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.584819 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.586220 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.586739 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.586868 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.587290 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.587578 (+ 288us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.587720 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.587847 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.587924 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":1240,"threads_started":1}
I20260926 07:45:03.590283 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.581380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.11.9.125:44967)
0926 07:45:03.581608 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.581616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.583428 (+ 1812us) server_negotiation.cc:407] Connection header received
0926 07:45:03.583865 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.583870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.583966 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.584021 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.584891 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.585713 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.587831 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.588529 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.588645 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.589433 (+ 788us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.589571 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.589673 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.589883 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.589979 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:45:03.591738 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:03.583518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.11.9.124:39589)
0926 07:45:03.583688 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:03.583693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:03.584173 (+ 480us) server_negotiation.cc:407] Connection header received
0926 07:45:03.584330 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:03.584338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:03.584433 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:03.584542 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:03.586549 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.587626 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.589782 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:03.590286 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:03.590427 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:03.591039 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:03.591137 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:03.591209 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:03.591366 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 07:45:03.591470 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 07:45:03.594576 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1232
I20260926 07:45:03.595713 1294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:03.704829 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1232
I20260926 07:45:03.725821 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1306
I20260926 07:45:03.726941 1368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:03.834134 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1306
I20260926 07:45:03.854257 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1380
I20260926 07:45:03.855413 1442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:03.962410 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1380
2026-09-26T07:45:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4375 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 07:45:04 dist-test-slave-x492 krb5kdc[1629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 07:45:04 dist-test-slave-x492 krb5kdc[1629](info): set up 2 sockets
Sep 26 07:45:04 dist-test-slave-x492 krb5kdc[1629](info): commencing operation
krb5kdc: starting...
W20260926 07:45:05.066123 11301 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/test-user.keytab.
Sep 26 07:45:05 dist-test-slave-x492 krb5kdc[1629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T07:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.9.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
Entry for principal kudu/127.11.9.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab.
I20260926 07:45:05.322713 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45651
--webserver_interface=127.11.9.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 07:45:05.641650 1642 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:05.641958 1642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:05.642022 1642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:05.642107 1642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:45:05.649072 1642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:05.649156 1642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:05.649286 1642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:05.649339 1642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:05.649401 1642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:05.658876 1642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.126.keytab
--principal=kudu/127.11.9.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.126:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.9.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:05.660043 1642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:05.661288 1642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:05.673029 1647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:05.673074 1650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:05.673075 1648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:05.673880 1642 server_base.cc:1034] running on GCE node
I20260926 07:45:05.674587 1642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:05.675930 1642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:05.677230 1642 hybrid_clock.cc:648] HybridClock initialized: now 1790408705677111 us; error 43 us; skew 500 ppm
I20260926 07:45:05.677570 1642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:05 dist-test-slave-x492 krb5kdc[1629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:05.686439 1642 init.cc:377] Logged in from keytab as kudu/127.11.9.126@KRBTEST.COM (short username kudu)
I20260926 07:45:05.687928 1642 webserver.cc:460] Webserver started at http://127.11.9.126:35335/ using document root <none> and password file <none>
I20260926 07:45:05.688594 1642 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:05.688680 1642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:05.688952 1642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:05.691871 1642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data/instance:
uuid: "f2d2e94eb59847198ec8df5851332416"
format_stamp: "Formatted at 2026-09-26 07:45:05 on dist-test-slave-x492"
server_key: "56a249dcefd0a328cbab56bdab1376c8"
server_key_iv: "8d71654d39c0bf009cd08df919b8dfb2"
server_key_version: "encryptionkey@0"
I20260926 07:45:05.692590 1642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance:
uuid: "f2d2e94eb59847198ec8df5851332416"
format_stamp: "Formatted at 2026-09-26 07:45:05 on dist-test-slave-x492"
server_key: "56a249dcefd0a328cbab56bdab1376c8"
server_key_iv: "8d71654d39c0bf009cd08df919b8dfb2"
server_key_version: "encryptionkey@0"
I20260926 07:45:05.697598 1642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260926 07:45:05.700878 1657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:05.702219 1642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:45:05.702399 1642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal
uuid: "f2d2e94eb59847198ec8df5851332416"
format_stamp: "Formatted at 2026-09-26 07:45:05 on dist-test-slave-x492"
server_key: "56a249dcefd0a328cbab56bdab1376c8"
server_key_iv: "8d71654d39c0bf009cd08df919b8dfb2"
server_key_version: "encryptionkey@0"
I20260926 07:45:05.702602 1642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:05.716151 1642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:05.720016 1642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:05.720283 1642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:05.739037 1642 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.126:45651
I20260926 07:45:05.739058 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.126:45651 every 8 connection(s)
I20260926 07:45:05.740346 1642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/data/info.pb
I20260926 07:45:05.744007 1709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:05.744096 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1642
I20260926 07:45:05.744306 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-0/wal/instance
I20260926 07:45:05.744747 11301 external_mini_cluster.cc:1428] Setting key 7c8863f6c5fa8902e1817c9781395ce2
I20260926 07:45:05.750479 1709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:05 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.9.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.125@KRBTEST.COM" created.
I20260926 07:45:05.772584 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.753638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.11.9.126:45651)
0926 07:45:05.754467 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:05.754481 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:05.754903 (+ 422us) server_negotiation.cc:407] Connection header received
0926 07:45:05.756834 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:05.756839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:05.757052 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:05.757287 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:05.758499 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:05.759652 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:05.761241 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:05.761782 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:05.761978 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:05.765579 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:05.765586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:05.765618 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:05.765649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:05.768480 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:05.769249 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:05.769261 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:05.769263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:05.769435 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:05.769945 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:05.769949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:05.769951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:05.770431 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:05.770600 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:05.771269 (+ 669us) server_negotiation.cc:299] Negotiation successful
0926 07:45:05.771538 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":575,"thread_start_us":380,"threads_started":1}
I20260926 07:45:05.772634 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.753638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45651 (local address 127.0.0.1:34222)
0926 07:45:05.754739 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.754779 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:05.756468 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:05.757289 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:05.757295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:05.757971 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:05.758288 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:05.758304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:05.759804 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:05.759809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:05.761075 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:05.761098 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:05.762021 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:05.762028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:05.762239 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:05.762920 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:05.762946 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:05.765366 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:05.768649 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:05.768659 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:05.768670 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:05.769057 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:05.769555 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:05.769560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:05.769563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:05.769801 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:05.770585 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:05.770595 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:05.770863 (+ 268us) client_negotiation.cc:769] Sending connection context
0926 07:45:05.771156 (+ 293us) client_negotiation.cc:240] Negotiation successful
0926 07:45:05.771729 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":74,"thread_start_us":698,"threads_started":1}
I20260926 07:45:05.775385 1709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:05.776340 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.775993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33132)
0926 07:45:05.776146 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.776241 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 07:45:05.776558 1659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
I20260926 07:45:05.778054 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.777720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33134)
0926 07:45:05.777871 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.777933 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 07:45:05.778239 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:33099: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
Entry for principal kudu/127.11.9.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab.
I20260926 07:45:05.800453 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:33099
--webserver_interface=127.11.9.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:45:05.827557 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } attempt: 1
I20260926 07:45:05.828769 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.828310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33138)
0926 07:45:05.828526 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.828640 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 07:45:05.829773 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.829419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33144)
0926 07:45:05.829564 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.829647 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 07:45:05.829943 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:33099: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
I20260926 07:45:05.898171 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } attempt: 2
I20260926 07:45:05.899401 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.898929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33158)
0926 07:45:05.899175 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.899265 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 07:45:05.900552 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.900147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33168)
0926 07:45:05.900327 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.900410 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 07:45:05.900738 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:33099: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
I20260926 07:45:05.991966 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } attempt: 3
I20260926 07:45:05.993656 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.992634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33170)
0926 07:45:05.993264 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.993372 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467}
I20260926 07:45:05.994899 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:05.994477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33182)
0926 07:45:05.994645 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:05.994731 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 07:45:05.995074 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:33099: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
W20260926 07:45:06.127929 1716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:06.128314 1716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:06.128444 1716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:06.128582 1716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:45:06.135673 1716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:06.135792 1716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:06.135896 1716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:06.135975 1716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:06.136065 1716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:06.138316 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } attempt: 4
I20260926 07:45:06.139572 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.139061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33186)
0926 07:45:06.139359 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.139442 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260926 07:45:06.140544 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.140204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33198)
0926 07:45:06.140357 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.140432 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 07:45:06.140777 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.125:33099: Network error: Client connection negotiation failed: client connection to 127.11.9.125:33099: connect: Connection refused (error 111)
I20260926 07:45:06.146492 1716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.125.keytab
--principal=kudu/127.11.9.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.125:33099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.9.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:06.147845 1716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:06.149143 1716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:06.160669 1721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:06.160705 1724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:06.160754 1722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:06.166728 1716 server_base.cc:1034] running on GCE node
I20260926 07:45:06.167590 1716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:06.168958 1716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:06.170241 1716 hybrid_clock.cc:648] HybridClock initialized: now 1790408706170180 us; error 61 us; skew 500 ppm
I20260926 07:45:06.170598 1716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:06.173858 1716 init.cc:377] Logged in from keytab as kudu/127.11.9.125@KRBTEST.COM (short username kudu)
I20260926 07:45:06.175325 1716 webserver.cc:460] Webserver started at http://127.11.9.125:32889/ using document root <none> and password file <none>
I20260926 07:45:06.176028 1716 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:06.176156 1716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:06.176487 1716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:06.179407 1716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data/instance:
uuid: "4ffa99b8accc42f984a36a901c8f463e"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "efdc9257eefd19ea3cc2f436f15dc6dd"
server_key_iv: "2ba4c16f5055b5eff1cc4ba22955901e"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.180193 1716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal/instance:
uuid: "4ffa99b8accc42f984a36a901c8f463e"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "efdc9257eefd19ea3cc2f436f15dc6dd"
server_key_iv: "2ba4c16f5055b5eff1cc4ba22955901e"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.185212 1716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 07:45:06.188488 1731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:06.189855 1716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 07:45:06.190052 1716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal
uuid: "4ffa99b8accc42f984a36a901c8f463e"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "efdc9257eefd19ea3cc2f436f15dc6dd"
server_key_iv: "2ba4c16f5055b5eff1cc4ba22955901e"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.190270 1716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:06.210041 1716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:06.214303 1716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:06.214586 1716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:06.233093 1716 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.125:33099
I20260926 07:45:06.233112 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.125:33099 every 8 connection(s)
I20260926 07:45:06.234628 1716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/data/info.pb
I20260926 07:45:06.238313 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:06.244103 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1716
I20260926 07:45:06.244302 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-1/wal/instance
I20260926 07:45:06.244761 11301 external_mini_cluster.cc:1428] Setting key c5f6b87dc4d733c016e8de1cdb77ecf7
I20260926 07:45:06.244654 1783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:45:06.264066 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.247327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.11.9.126:45651)
0926 07:45:06.247543 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.247548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.248755 (+ 1207us) server_negotiation.cc:407] Connection header received
0926 07:45:06.250664 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.250669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.250783 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.250899 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.252329 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.253016 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.254587 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.254990 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.255136 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.258484 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.258489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.258492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.258525 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.261067 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.261852 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.261859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.261868 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.261982 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.262398 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.262403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.262408 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.262685 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.262842 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.263516 (+ 674us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.263742 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 07:45:06.264997 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.247816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45651 (local address 127.0.0.1:34224)
0926 07:45:06.248594 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.248635 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.250399 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.251125 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.251133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.251634 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.252166 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.252182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.253221 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.253229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.254425 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.254436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.255224 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.255232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.255451 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.256135 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.256156 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.258265 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.261275 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.261282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.261296 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.261712 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.262100 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.262105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.262108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.262280 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.262821 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.262828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.263144 (+ 316us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.263404 (+ 260us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.263868 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":361,"threads_started":1}
I20260926 07:45:06.266877 1783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.9.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.9.124@KRBTEST.COM" created.
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:06.282002 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.267837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.11.9.125:33099)
0926 07:45:06.268533 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.268539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.268564 (+ 25us) server_negotiation.cc:407] Connection header received
0926 07:45:06.268612 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.268618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.268778 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.268891 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.269682 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.270746 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.271936 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.272439 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.272579 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.275459 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.275473 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.275491 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.275532 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.279008 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.279834 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.279840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.279843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.279961 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.280377 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.280380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.280382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.280725 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.280873 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.281425 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.281674 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":516,"thread_start_us":366,"threads_started":1}
I20260926 07:45:06.282052 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.267569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33212)
0926 07:45:06.267772 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.267790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.267890 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.268913 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.268916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.269323 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.269454 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.269462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.270933 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.270936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.271794 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.271805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.272602 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.272607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.272771 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.273346 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.273362 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.275247 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.279154 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.279160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.279165 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.279693 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.280075 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.280093 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.280099 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.280275 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.280851 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.280859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.281037 (+ 178us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.281243 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.281559 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 07:45:06.284500 1783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:06.285629 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.285220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50514)
0926 07:45:06.285387 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.285491 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 07:45:06.285890 1735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.287372 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.287073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50524)
0926 07:45:06.287208 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.287281 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 07:45:06.287672 1783 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
Entry for principal kudu/127.11.9.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab.
I20260926 07:45:06.303781 11301 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
/tmp/dist-test-taskm0hQhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:42841
--webserver_interface=127.11.9.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.9.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 07:45:06.336962 1783 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 1
I20260926 07:45:06.338214 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.337662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50526)
0926 07:45:06.337889 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.338019 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 07:45:06.339232 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.338845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50528)
0926 07:45:06.338998 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.339115 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 07:45:06.339448 1783 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.407684 1783 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 2
I20260926 07:45:06.408900 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.408365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50536)
0926 07:45:06.408596 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.408712 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 07:45:06.410017 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.409625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50544)
0926 07:45:06.409803 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.409885 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 07:45:06.410219 1783 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.440022 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } attempt: 5
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:06.455018 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.441025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.11.9.125:33099)
0926 07:45:06.441256 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.441261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.441280 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:45:06.441397 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.441400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.441485 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.441650 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.442663 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.443329 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.445001 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.445451 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.445540 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.449419 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.449423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.449426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.449459 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.452282 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.453012 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.453016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.453018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.453132 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.453598 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.453603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.453606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.453880 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.454042 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.454419 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.454693 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 07:45:06.455019 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.440748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33218)
0926 07:45:06.441015 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.441054 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.441240 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.441706 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.441713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.442258 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.442504 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.442518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.443466 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.443472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.444848 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.444863 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.445595 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.445601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.445814 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.446474 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.446503 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.449230 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.452413 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.452418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.452421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.452899 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.453292 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.453297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.453301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.453448 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.453998 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.454003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.454168 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.454313 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.454620 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 07:45:06.456066 1709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:45:06.457056 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.456673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50558)
0926 07:45:06.456836 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.456925 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 07:45:06.458022 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.457717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50574)
0926 07:45:06.457862 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.457933 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 07:45:06.458228 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.501453 1783 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 3
I20260926 07:45:06.502715 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.502152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50586)
0926 07:45:06.502397 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.502523 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 07:45:06.503939 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.503465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50592)
0926 07:45:06.503704 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.503797 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260926 07:45:06.504235 1783 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.509415 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 1
I20260926 07:45:06.510388 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.509962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50604)
0926 07:45:06.510145 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.510242 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 07:45:06.511379 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.511079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50608)
0926 07:45:06.511223 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.511283 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 07:45:06.511651 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.579876 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 2
I20260926 07:45:06.581013 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.580519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50618)
0926 07:45:06.580751 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.580848 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 07:45:06.582054 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.581748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50626)
0926 07:45:06.581889 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.581960 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 07:45:06.582281 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
W20260926 07:45:06.628427 1790 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 07:45:06.628752 1790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:45:06.628883 1790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:45:06.628998 1790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:45:06.636277 1790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:45:06.636361 1790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 07:45:06.636417 1790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:45:06.636643 1790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:45:06.636780 1790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:45:06.646646 1790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.9.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.9.126:45651,127.11.9.125:33099,127.11.9.124:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/krb5kdc/kudu_127.11.9.124.keytab
--principal=kudu/127.11.9.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.9.124:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.9.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:26:09 UTC on ccycloud.quasar-arzrrx.root.comops.site
build id 4673740
ASAN enabled
I20260926 07:45:06.647475 1783 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 4
I20260926 07:45:06.648034 1790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:45:06.648782 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.648213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50632)
0926 07:45:06.648485 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.648603 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 07:45:06.649834 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.649446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50634)
0926 07:45:06.649618 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.649705 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 07:45:06.649859 1790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:45:06.650068 1783 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
W20260926 07:45:06.661755 1796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:06.661793 1795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:45:06.661947 1798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:45:06.662618 1790 server_base.cc:1034] running on GCE node
I20260926 07:45:06.663311 1790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:45:06.664600 1790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:45:06.665853 1790 hybrid_clock.cc:648] HybridClock initialized: now 1790408706665799 us; error 65 us; skew 500 ppm
I20260926 07:45:06.666175 1790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 07:45:06.669354 1790 init.cc:377] Logged in from keytab as kudu/127.11.9.124@KRBTEST.COM (short username kudu)
I20260926 07:45:06.670801 1790 webserver.cc:460] Webserver started at http://127.11.9.124:43689/ using document root <none> and password file <none>
I20260926 07:45:06.671455 1790 fs_manager.cc:359] Metadata directory not provided
I20260926 07:45:06.671586 1790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:45:06.671895 1790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:45:06.674830 1790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data/instance:
uuid: "3c98f1c7a00d4081a1d3390ba4296c84"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "ecfd8c58cbd7166b50c156d26f067647"
server_key_iv: "3b014e6923025900ddc9217591f4d7af"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.675633 1790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal/instance:
uuid: "3c98f1c7a00d4081a1d3390ba4296c84"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "ecfd8c58cbd7166b50c156d26f067647"
server_key_iv: "3b014e6923025900ddc9217591f4d7af"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.680562 1790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 07:45:06.684005 1805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:45:06.685395 1790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 07:45:06.685559 1790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data,/tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal
uuid: "3c98f1c7a00d4081a1d3390ba4296c84"
format_stamp: "Formatted at 2026-09-26 07:45:06 on dist-test-slave-x492"
server_key: "ecfd8c58cbd7166b50c156d26f067647"
server_key_iv: "3b014e6923025900ddc9217591f4d7af"
server_key_version: "encryptionkey@0"
I20260926 07:45:06.685772 1790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:45:06.688495 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 3
I20260926 07:45:06.689615 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.689141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50636)
0926 07:45:06.689340 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.689419 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 07:45:06.690690 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.690286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50644)
0926 07:45:06.690512 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.690590 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260926 07:45:06.690966 1709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.9.124:42841: Network error: Client connection negotiation failed: client connection to 127.11.9.124:42841: connect: Connection refused (error 111)
I20260926 07:45:06.698679 1790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:45:06.702665 1790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:45:06.702924 1790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:45:06.721683 1790 rpc_server.cc:307] RPC server started. Bound to: 127.11.9.124:42841
I20260926 07:45:06.721702 1856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.9.124:42841 every 8 connection(s)
I20260926 07:45:06.723066 1790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/data/info.pb
I20260926 07:45:06.726246 11301 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu as pid 1790
I20260926 07:45:06.726420 11301 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm0hQhn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790408351541924-11301-0/minicluster-data/master-2/wal/instance
I20260926 07:45:06.726972 1857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:45:06.726984 11301 external_mini_cluster.cc:1428] Setting key c6d7a672e1fd3c417aeb7cf8452c5c6d
I20260926 07:45:06.735338 1857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
I20260926 07:45:06.749940 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.728734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.11.9.124:42841)
0926 07:45:06.729514 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.729525 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.729563 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:45:06.731533 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.731552 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.731926 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.732324 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.733829 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.735044 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.736358 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.736995 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.737234 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.742243 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.742254 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.742269 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.742315 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.745900 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.746576 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.746584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.746592 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.746823 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.747234 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.747239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.747242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.747863 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.748180 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.748558 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.748852 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":390,"threads_started":1}
I20260926 07:45:06.754136 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.737788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.11.9.126:45651)
0926 07:45:06.737979 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.737987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.738666 (+ 679us) server_negotiation.cc:407] Connection header received
0926 07:45:06.738976 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.738982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.739067 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.739176 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.740595 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.741374 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.743080 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.743858 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.743995 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.748209 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.748216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.748220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.748257 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.751194 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.751970 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.751979 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.751984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.752173 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.752565 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.752573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.752579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.752918 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.753086 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.753638 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.753789 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 07:45:06.754285 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.737716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.126:45651 (local address 127.0.0.1:34238)
0926 07:45:06.738533 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.738576 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.738784 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.739267 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.739276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.740075 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.740428 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.740443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.741519 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.741524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.742887 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.742904 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.744333 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.744344 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.744554 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.745229 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.745263 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.748031 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.751311 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.751320 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.751326 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.751845 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.752273 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.752278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.752281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.752435 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.753068 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.753075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.753322 (+ 247us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.753549 (+ 227us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.753851 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":382,"threads_started":1}
I20260926 07:45:06.755757 1857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.126@KRBTEST.COM
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:06.768721 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.752835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.11.9.126:45651)
0926 07:45:06.753683 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.753689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.753711 (+ 22us) server_negotiation.cc:407] Connection header received
0926 07:45:06.753755 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.753761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.753834 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.753949 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.755412 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.756512 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.758216 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.758607 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.758737 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.762841 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.762846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.762849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.762877 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.766136 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.766744 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.766749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.766753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.766905 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.767269 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.767274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.767277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.767528 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.767808 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.768094 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.768309 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":476,"threads_started":1}
I20260926 07:45:06.769084 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.756329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.125:33099 (local address 127.0.0.1:33226)
0926 07:45:06.756545 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.756564 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.756647 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.756930 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.756934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.757385 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.757567 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.757578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.758512 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.758519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.759546 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.759557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.760270 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.760276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.760464 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.761040 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.761055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.763324 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.766838 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.766841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.766844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.767179 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.767556 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.767561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.767564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.767727 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.768223 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.768228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.768373 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.768538 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.768739 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 07:45:06.769269 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.756501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.11.9.125:33099)
0926 07:45:06.756661 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.756665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.756703 (+ 38us) server_negotiation.cc:407] Connection header received
0926 07:45:06.756750 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.756754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.756808 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.756880 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.757714 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.758376 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.759701 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.760118 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.760246 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.763566 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.763575 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.763581 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.763628 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.766729 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.767299 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.767306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.767309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.767441 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.767840 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.767845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.767849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.768109 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.768260 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.768665 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.768892 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 07:45:06.770314 1857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.124@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.9.125@KRBTEST.COM
I20260926 07:45:06.785097 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.771153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50670)
0926 07:45:06.771330 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.771349 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.771446 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.771717 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.771723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.772202 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.772368 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.772378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.773792 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.773797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.775167 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.775182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.776125 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.776129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.776380 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.776931 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.776958 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.779330 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.782560 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.782566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.782571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.783047 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.783382 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.783387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.783390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.783556 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.784009 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.784014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.784200 (+ 186us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.784357 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.784632 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 07:45:06.785192 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.771324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.11.9.124:42841)
0926 07:45:06.771472 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.771476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.771492 (+ 16us) server_negotiation.cc:407] Connection header received
0926 07:45:06.771528 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.771531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.771580 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.771650 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.772535 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.773651 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.775310 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.775944 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.776095 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.779488 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.779493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.779496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.779522 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.782430 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.783161 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.783164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.783166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.783271 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.783659 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.783662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.783664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.783892 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.783995 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.784493 (+ 498us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.784795 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":49}
I20260926 07:45:06.786559 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.770628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.11.9.125:33099)
0926 07:45:06.770798 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.770803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.770818 (+ 15us) server_negotiation.cc:407] Connection header received
0926 07:45:06.770871 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.770874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.770921 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.771017 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.772581 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.773669 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.775411 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.776089 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.776198 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.780765 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.780770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.780773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.780801 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.783661 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.784418 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.784423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.784426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.784582 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.785001 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.785007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.785011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.785389 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.785676 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.785994 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.786227 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 07:45:06.790824 1857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: Bootstrap starting.
I20260926 07:45:06.791201 11301 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 07:45:06.794368 1857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:06.795672 1857 log.cc:824] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:06.793478 1684 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f2d2e94eb59847198ec8df5851332416"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34244
I20260926 07:45:06.798841 1857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: No bootstrap required, opened a new log
I20260926 07:45:06.809460 1857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.809839 1857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c98f1c7a00d4081a1d3390ba4296c84, State: Initialized, Role: FOLLOWER
I20260926 07:45:06.810326 1857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.811406 1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:06.811760 1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:06.812440 1857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:45:06.821779 1875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:45:06.821906 1875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:45:06.819774 1790 master_runner.cc:186] Error getting master registration for 127.11.9.126:45651: OK, instance_id { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" instance_seqno: 1790408705733768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:45:06.830837 1790 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:45:06.868232 1709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 4
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408705, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.126@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
I20260926 07:45:06.884473 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.869033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50676)
0926 07:45:06.869276 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.869299 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.869435 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.869957 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.869962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.870471 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.870704 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.870717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.872187 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.872193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.873707 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.873722 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.874710 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.874716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.874907 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.875589 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.875615 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.878524 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.881739 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.881746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.881752 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.882237 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.882641 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.882646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.882650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.882810 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.883425 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.883431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.883636 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.883786 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.884074 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 07:45:06.884490 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.869456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.11.9.124:42841)
0926 07:45:06.869668 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.869673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.869691 (+ 18us) server_negotiation.cc:407] Connection header received
0926 07:45:06.869748 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.869752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.869810 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.869895 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.870859 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.872024 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.873842 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.874552 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.874675 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.878737 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.878742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.878745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.878776 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.881622 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.882363 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.882368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.882374 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.882517 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.882923 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.882928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.882931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.883284 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.883451 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.883911 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.884129 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:45:06.890609 1709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: Bootstrap starting.
I20260926 07:45:06.894698 1709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:06.895951 1709 log.cc:824] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:06.899191 1709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: No bootstrap required, opened a new log
I20260926 07:45:06.906895 1709 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.907295 1709 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d2e94eb59847198ec8df5851332416, State: Initialized, Role: FOLLOWER
I20260926 07:45:06.907801 1709 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.908778 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:06.909190 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:06.910068 1709 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:45:06.910132 1684 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 07:45:06.910346 1684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:06.912954 1684 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.914865 1684 leader_election.cc:288] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [CANDIDATE]: Term 1 election: Requested vote from peers 4ffa99b8accc42f984a36a901c8f463e (127.11.9.125:33099), 3c98f1c7a00d4081a1d3390ba4296c84 (127.11.9.124:42841)
I20260926 07:45:06.915231 1832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d2e94eb59847198ec8df5851332416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c98f1c7a00d4081a1d3390ba4296c84"
I20260926 07:45:06.915717 1832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:06.919123 1832 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d2e94eb59847198ec8df5851332416 in term 1.
I20260926 07:45:06.919865 1661 leader_election.cc:302] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c98f1c7a00d4081a1d3390ba4296c84, f2d2e94eb59847198ec8df5851332416; no voters:
I20260926 07:45:06.920321 1878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:45:06.925498 1878 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [term 1 LEADER]: Becoming Leader. State: Replica: f2d2e94eb59847198ec8df5851332416, State: Running, Role: LEADER
I20260926 07:45:06.926429 1878 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:06.934863 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d2e94eb59847198ec8df5851332416. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:06.935269 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:06.948565 1891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:45:06.949281 1783 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } attempt: 5
W20260926 07:45:06.955578 1811 tablet.cc:2369] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 07:45:06 dist-test-slave-x492 krb5kdc[1629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790408706, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.9.125@KRBTEST.COM for kudu/127.11.9.124@KRBTEST.COM
W20260926 07:45:06.973342 1663 tablet.cc:2369] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:45:06.979557 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.949905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.9.124:42841 (local address 127.0.0.1:50682)
0926 07:45:06.950288 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0926 07:45:06.950309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 07:45:06.950439 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:45:06.951444 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 07:45:06.951448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 07:45:06.951947 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 07:45:06.952146 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.952158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.953742 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.953748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.955203 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 07:45:06.955219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.957985 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.957995 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 07:45:06.958297 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.959134 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 07:45:06.959193 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0926 07:45:06.964147 (+ 4954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 07:45:06.971870 (+ 7723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.971900 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.971907 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.972832 (+ 925us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.973796 (+ 964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 07:45:06.973824 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 07:45:06.973827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 07:45:06.974084 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 07:45:06.975397 (+ 1313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 07:45:06.975428 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 07:45:06.975715 (+ 287us) client_negotiation.cc:769] Sending connection context
0926 07:45:06.975984 (+ 269us) client_negotiation.cc:240] Negotiation successful
0926 07:45:06.976630 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260926 07:45:06.981473 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.950765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.11.9.124:42841)
0926 07:45:06.950943 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.950948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.950967 (+ 19us) server_negotiation.cc:407] Connection header received
0926 07:45:06.951028 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.951033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.951106 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.951198 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.952361 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.953547 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.956614 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.957552 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:06.957704 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:06.964449 (+ 6745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:06.964458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:06.964466 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:06.964516 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:06.971642 (+ 7126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.973105 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.973113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.973118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.973418 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:06.974414 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:06.974422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:06.974428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:06.975161 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:06.976899 (+ 1738us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:06.977718 (+ 819us) server_negotiation.cc:299] Negotiation successful
0926 07:45:06.979308 (+ 1590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 07:45:06.984601 1891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:45:06.987440 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: Bootstrap starting.
I20260926 07:45:06.994401 1783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: Neither blocks nor log segments found. Creating new log.
I20260926 07:45:06.998383 1783 log.cc:824] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:45:07.007323 1832 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [term 1 FOLLOWER]: Refusing update from remote peer f2d2e94eb59847198ec8df5851332416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:45:07.008409 1886 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:45:07.010504 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: No bootstrap required, opened a new log
I20260926 07:45:07.034950 1783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:07.035969 1783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffa99b8accc42f984a36a901c8f463e, State: Initialized, Role: FOLLOWER
I20260926 07:45:07.037127 1783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } }
I20260926 07:45:07.038134 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.950314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.11.9.125:33099)
0926 07:45:06.950546 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.950555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.993645 (+ 43090us) server_negotiation.cc:407] Connection header received
0926 07:45:06.993938 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.993947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.994050 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.994180 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:06.996266 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:06.997768 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.001350 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:07.001937 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.002045 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:07.023139 (+ 21094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:07.023147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:07.023152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:07.023199 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:07.027549 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.033952 (+ 6403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.033958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.033961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.034180 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.035828 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.035835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.035839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.036230 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:07.036471 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:07.037507 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0926 07:45:07.037766 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 07:45:07.043318 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.955764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.11.9.124:42841)
0926 07:45:06.974007 (+ 18243us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.974015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.000791 (+ 26776us) server_negotiation.cc:407] Connection header received
0926 07:45:07.000963 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.000968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.001068 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.001196 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.016396 (+ 15200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:07.017732 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.020028 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:07.020770 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.020875 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:07.030150 (+ 9275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:07.030157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:07.030162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:07.030208 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:07.035348 (+ 5140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.040404 (+ 5056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.040410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.040414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.040596 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.041013 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.041018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.041022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.041422 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:07.041678 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:07.042665 (+ 987us) server_negotiation.cc:299] Negotiation successful
0926 07:45:07.042897 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18108,"thread_start_us":18271,"threads_started":1}
I20260926 07:45:07.049840 1758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d2e94eb59847198ec8df5851332416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4ffa99b8accc42f984a36a901c8f463e"
I20260926 07:45:07.050652 1757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:45:07.054284 1757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [term 1 FOLLOWER]: Refusing update from remote peer f2d2e94eb59847198ec8df5851332416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:45:07.047988 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.054822 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:07.055294 1886 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:45:07.060271 1758 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d2e94eb59847198ec8df5851332416 in term 1.
I20260926 07:45:07.061990 1783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:45:07.064610 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:06.949198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.11.9.126:45651)
0926 07:45:06.949410 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:06.949417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:06.968551 (+ 19134us) server_negotiation.cc:407] Connection header received
0926 07:45:06.977949 (+ 9398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:06.977956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:06.978067 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:06.978141 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.001723 (+ 23582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:07.002844 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.046380 (+ 43536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:07.047117 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:07.047223 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:07.055912 (+ 8689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 07:45:07.055921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 07:45:07.055928 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 07:45:07.055968 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 07:45:07.059674 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.061292 (+ 1618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.061297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.061300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.061449 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 07:45:07.062205 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 07:45:07.062213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 07:45:07.062216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 07:45:07.062571 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 07:45:07.062796 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:07.063809 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0926 07:45:07.064130 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 07:45:07.076011 1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d2e94eb59847198ec8df5851332416. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.076357 1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:07.088443 1899 mvcc.cc:205] Tried to move back new op lower bound from 7333514063876907008 to 7333514063579602944. Current Snapshot: MvccSnapshot[applied={T|T < 7333514063876907008}]
I20260926 07:45:07.089048 1902 mvcc.cc:205] Tried to move back new op lower bound from 7333514063876907008 to 7333514063579602944. Current Snapshot: MvccSnapshot[applied={T|T < 7333514063876907008}]
I20260926 07:45:07.079428 1642 master_runner.cc:186] Error getting master registration for 127.11.9.125:33099: OK, instance_id { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" instance_seqno: 1790408706227769 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:45:07.092525 1642 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:45:07.093088 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.093333 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:07.094769 1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.094964 1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:07.095449 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d2e94eb59847198ec8df5851332416. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.095785 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:07.097373 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.097608 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416 [sys.catalog]: This master's current role is: LEADER
I20260926 07:45:07.098826 1907 mvcc.cc:205] Tried to move back new op lower bound from 7333514063876907008 to 7333514063579602944. Current Snapshot: MvccSnapshot[applied={T|T < 7333514063876907008}]
I20260926 07:45:07.105432 1891 catalog_manager.cc:1347] Generated new cluster ID: 891dd06f2f5145b7b31c57befddf6446
I20260926 07:45:07.105540 1891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:45:07.107455 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2d2e94eb59847198ec8df5851332416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d2e94eb59847198ec8df5851332416" member_type: VOTER last_known_addr { host: "127.11.9.126" port: 45651 } } peers { permanent_uuid: "4ffa99b8accc42f984a36a901c8f463e" member_type: VOTER last_known_addr { host: "127.11.9.125" port: 33099 } } peers { permanent_uuid: "3c98f1c7a00d4081a1d3390ba4296c84" member_type: VOTER last_known_addr { host: "127.11.9.124" port: 42841 } } }
I20260926 07:45:07.107677 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:45:07.132792 1891 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:45:07.136551 1891 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:45:07.153898 1891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2d2e94eb59847198ec8df5851332416: Generated new TSK 0
I20260926 07:45:07.154989 1891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:45:07.159957 1921 catalog_manager.cc:1259] Loaded cluster ID: 891dd06f2f5145b7b31c57befddf6446
I20260926 07:45:07.160230 1921 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: loading cluster ID for follower catalog manager: success
I20260926 07:45:07.163741 1921 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: acquiring CA information for follower catalog manager: success
W20260926 07:45:07.165017 1921 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260926 07:45:07.263935 1862 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.255535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.11.9.126:45651)
0926 07:45:07.255710 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.255716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.261650 (+ 5934us) server_negotiation.cc:407] Connection header received
0926 07:45:07.261957 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.261963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.262069 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.262150 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.263565 (+ 1415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34264: BlockingRecv error: recv got EOF from 127.0.0.1:34264 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 07:45:07.272074 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.261991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.11.9.125:33099)
0926 07:45:07.262230 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.262237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.269628 (+ 7391us) server_negotiation.cc:407] Connection header received
0926 07:45:07.270383 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.270396 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.270523 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.270608 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.271766 (+ 1158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33266: BlockingRecv error: recv got EOF from 127.0.0.1:33266 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 07:45:07.275116 1896 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.264981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.11.9.124:42841)
0926 07:45:07.265155 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.265179 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.272853 (+ 7674us) server_negotiation.cc:407] Connection header received
0926 07:45:07.273760 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.273767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.273843 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.273907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.274892 (+ 985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50704: BlockingRecv error: recv got EOF from 127.0.0.1:50704 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 07:45:07.341461 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.328424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.11.9.125:33099)
0926 07:45:07.328620 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.328626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.339252 (+ 10626us) server_negotiation.cc:407] Connection header received
0926 07:45:07.339821 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.339827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.339958 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.340042 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.341119 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33274: BlockingRecv error: recv got EOF from 127.0.0.1:33274 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 07:45:07.390215 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.376831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.11.9.125:33099)
0926 07:45:07.377052 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.377058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.388053 (+ 10995us) server_negotiation.cc:407] Connection header received
0926 07:45:07.388600 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.388606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.388709 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.388791 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.389937 (+ 1146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33284: BlockingRecv error: recv got EOF from 127.0.0.1:33284 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 07:45:07.444490 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.430255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.11.9.125:33099)
0926 07:45:07.430459 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.430466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.442326 (+ 11860us) server_negotiation.cc:407] Connection header received
0926 07:45:07.442889 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.442895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.443023 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.443111 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.444210 (+ 1099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33292: BlockingRecv error: recv got EOF from 127.0.0.1:33292 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 07:45:07.522505 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.513151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.11.9.125:33099)
0926 07:45:07.513433 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.513441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.520483 (+ 7042us) server_negotiation.cc:407] Connection header received
0926 07:45:07.520985 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.520992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.521114 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.521217 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.522267 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33296: BlockingRecv error: recv got EOF from 127.0.0.1:33296 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
W20260926 07:45:07.595748 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.586806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.11.9.125:33099)
0926 07:45:07.587001 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.587007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.593830 (+ 6823us) server_negotiation.cc:407] Connection header received
0926 07:45:07.594174 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.594181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.594273 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.594362 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.595487 (+ 1125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33302: BlockingRecv error: recv got EOF from 127.0.0.1:33302 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 07:45:07.629128 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.626850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.11.9.125:33099)
0926 07:45:07.627083 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.627092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.627672 (+ 580us) server_negotiation.cc:407] Connection header received
0926 07:45:07.627908 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.627912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.627981 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.628116 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.628856 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33318: BlockingRecv error: recv got EOF from 127.0.0.1:33318 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 07:45:07.666563 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.664685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.11.9.125:33099)
0926 07:45:07.664897 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.664905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.665180 (+ 275us) server_negotiation.cc:407] Connection header received
0926 07:45:07.665397 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.665403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.665502 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.665656 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.666313 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33332: BlockingRecv error: recv got EOF from 127.0.0.1:33332 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 07:45:07.735802 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.733610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33338 (local address 127.11.9.125:33099)
0926 07:45:07.733859 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.733868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.734274 (+ 406us) server_negotiation.cc:407] Connection header received
0926 07:45:07.734473 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.734477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.734620 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.734735 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.735531 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33338: BlockingRecv error: recv got EOF from 127.0.0.1:33338 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 07:45:07.823978 1875 catalog_manager.cc:1259] Loaded cluster ID: 891dd06f2f5145b7b31c57befddf6446
I20260926 07:45:07.824084 1875 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: loading cluster ID for follower catalog manager: success
I20260926 07:45:07.827263 1875 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: acquiring CA information for follower catalog manager: success
I20260926 07:45:07.828945 1875 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c98f1c7a00d4081a1d3390ba4296c84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 07:45:07.868889 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:07.867157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.11.9.125:33099)
0926 07:45:07.867332 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:07.867336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:07.867629 (+ 293us) server_negotiation.cc:407] Connection header received
0926 07:45:07.867850 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:07.867855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:07.867949 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:07.868053 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:07.868681 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33350: BlockingRecv error: recv got EOF from 127.0.0.1:33350 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 07:45:08.132951 1787 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:45:08.131308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.11.9.125:33099)
0926 07:45:08.131516 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.131524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.131681 (+ 157us) server_negotiation.cc:407] Connection header received
0926 07:45:08.131851 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.131854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.131928 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.132010 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 07:45:08.132753 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33364: BlockingRecv error: recv got EOF from 127.0.0.1:33364 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 07:45:08.179793 1921 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4ffa99b8accc42f984a36a901c8f463e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:45:08.656816 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:08.650913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.11.9.125:33099)
0926 07:45:08.651435 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.651442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.651541 (+ 99us) server_negotiation.cc:407] Connection header received
0926 07:45:08.651768 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.651772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.651920 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.652058 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:08.652499 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.653245 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.654620 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.655052 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.655189 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:08.655676 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:08.656187 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:08.656344 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:08.656444 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 07:45:08.656533 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":280,"threads_started":1}
I20260926 07:45:08.669000 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:08.662452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.11.9.124:42841)
0926 07:45:08.663197 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.663206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.663229 (+ 23us) server_negotiation.cc:407] Connection header received
0926 07:45:08.663354 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.663361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.663572 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.663715 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:08.664127 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.664928 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.666507 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.666956 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.667071 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:08.667616 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:08.668154 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:08.668280 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:08.668512 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 07:45:08.668719 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":465,"threads_started":1}
I20260926 07:45:08.682300 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:08.675620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.11.9.125:33099)
0926 07:45:08.675842 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.675850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.676777 (+ 927us) server_negotiation.cc:407] Connection header received
0926 07:45:08.676977 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.676985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.677054 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.677152 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:08.677772 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.678767 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.680267 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.680893 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.681006 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:08.681628 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:08.681737 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:08.681848 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:08.681967 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 07:45:08.682040 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 07:45:08.682408 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:08.674806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.11.9.124:42841)
0926 07:45:08.675010 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.675018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.676211 (+ 1193us) server_negotiation.cc:407] Connection header received
0926 07:45:08.676404 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.676408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.676484 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.676542 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:08.677355 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.678214 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.680333 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.681005 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.681123 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:08.681666 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:08.681806 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:08.681867 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:08.682064 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 07:45:08.682168 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 07:45:08.683305 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 07:45:08.674705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.11.9.126:45651)
0926 07:45:08.676280 (+ 1575us) server_negotiation.cc:206] Beginning negotiation
0926 07:45:08.676288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 07:45:08.676308 (+ 20us) server_negotiation.cc:407] Connection header received
0926 07:45:08.676420 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 07:45:08.676424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 07:45:08.676637 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 07:45:08.676799 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 07:45:08.677526 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.678603 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.680665 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 07:45:08.681283 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 07:45:08.681409 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 07:45:08.681974 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 07:45:08.682635 (+ 661us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 07:45:08.682738 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 07:45:08.682912 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 07:45:08.683003 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1457,"thread_start_us":1352,"threads_started":1}
I20260926 07:45:08.687400 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1642
I20260926 07:45:08.688761 1704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:08.800640 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1642
I20260926 07:45:08.821300 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1716
I20260926 07:45:08.822430 1778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:08.930418 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1716
I20260926 07:45:08.950412 11301 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu pid 1790
I20260926 07:45:08.951609 1852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 07:45:09.058441 11301 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm0hQhn/build/asan/bin/kudu with pid 1790
2026-09-26T07:45:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5092 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9468 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (357484 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST